GH-22232: [C++][Python] Introduce optional default_column_type parameter

[vladborovtsov]

Rationale for this change

Add an optional default_column_type parameter to the CSV reading API (C++ and Python) to provide a fallback type when per-column types aren’t specified, improving schema consistency and complementing the existing column_types logic.

What changes are included in this PR?

• c++: new convert option "default_column_type" to augment logic around column_types parameter
• 3 reader tests: DefaultColumnTypePartialDefault, DefaultColumnTypeAllStringsWithHeader, DefaultColumnTypeAllStringsNoHeader). The last two tests are inspired by PARTIAL FIX: Improve leading zeros preservation with dtype=str for dict-based dtypes pandas-dev/pandas#62242 and BUG: pyarrow stripping leading zeros with dtype=str pandas-dev/pandas#57666
• python: corresponding changes to make cpp change consumable from python
• python: extended test_convert_options test - include. default_column_type
• python: added new test "test_default_column_type" which tests how the field impacts schema; also test implicitly verifies leading zero preservation
• relevant documentation update for python component;

Are these changes tested?

Yes. Existing and new tests are passing.

C++:

> [==========] Running 3 tests from 1 test suite.
> [----------] Global test environment set-up.
> [----------] 3 tests from ReaderTests
> [ RUN      ] ReaderTests.DefaultColumnTypePartialDefault
> [       OK ] ReaderTests.DefaultColumnTypePartialDefault (3 ms)
> [ RUN      ] ReaderTests.DefaultColumnTypeAllStringsWithHeader
> [       OK ] ReaderTests.DefaultColumnTypeAllStringsWithHeader (0 ms)
> [ RUN      ] ReaderTests.DefaultColumnTypeAllStringsNoHeader
> [       OK ] ReaderTests.DefaultColumnTypeAllStringsNoHeader (0 ms)
> [----------] 3 tests from ReaderTests (4 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 3 tests from 1 test suite ran. (4 ms total)
> [  PASSED  ] 3 tests.

All:

> [==========] 264 tests from 46 test suites ran. (452 ms total)
> [  PASSED  ] 264 tests.

pyarrow:
New tests are passing.

Are there any user-facing changes?

I believe this change is backward compatible. Parameter is optional and its default value doesn't change the existing behavior; All the existing rests are passing.

Maybe relevant: #22232

Relates to #47502

• GitHub Issue: [C++] CSV reader: add a default column type (or sentinel mapping) to avoid per-column enumeration #47502

• GitHub Issue: [C++] CSV reader: Ability to not infer column types. #22232

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[vladborovtsov]

@github-actions crossbow submit preview-docs

[github-actions]

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/18062577036

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[github-actions]

⚠️ GitHub issue #47502 has been automatically assigned in GitHub to PR creator.

[AlenkaF]

Thank you @vladborovtsov for the contribution.
I will add info about the proposed solution in the original issue (#22232) so I can see opinions from C++ devs on the proposed solution.

[github-actions]

⚠️ GitHub issue #22232 has been automatically assigned in GitHub to PR creator.

[vladborovtsov]

Hi @AlenkaF
I'm happy to continue the labour and discussion to get that merged.
As for AI, it wasn't used much, although I tried :) With such huge codebase the generation quality is quite low.

[AlenkaF]

Happy to see a response!
All good, it is totally OK to use gen AI wisely ;)

I will wait for an opinion from a C++ dev and in the meantime try to look at the Python part.