`DataScan` `count` method does not respect limit

[jayceslesar]

Apache Iceberg version

0.9.1 (latest release)

Please describe the bug 🐞

When calling count() on a DataScan, limit is not respected. Seems trivial but if I set a limit of 5 I expect 5 or less rows back, at least with a scan-like implementation

The underlying ArrowScan does not get passed the limit param

https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/__init__.py#L1940

This results in scans taking longer due to not respecting the limit.

The fix will involve more than just passing the limit to the ArrowScan

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[tusharchou]

Oh this seems important, i will groom this issue

[rashampreet-singh]

HI @jayceslesar I would like to work on this issue.

[tushar-choudhary-tc]

hi @rashampreet-singh how can I help you get unblocked

[tushar-choudhary-tc]

#1388

Here is the PR that where I had contributed this function.

[Fokko]

What kind of use-case would this serve? If the count is less than the number of rows in the table, then it always will be count == limit?

[kaustuvnandy]

Hi Fokko, tried to raise a small PR to solve for this.
Should I add another test case to re-create this? Can you please help me here!

[jayceslesar]

@Fokko I my use case is a short circuit to test for data existence

[jayceslesar]

although I suppose I can achieve this by simply performing a normal table scan with a limit...