feat(reading-order): ReadingOrder::Structure - fix tagged table order from the structure tree

[ajbufort]

Description

extract_spans_with_reading_order currently offers only geometric orders — TopToBottom and ColumnAware. So a Tagged PDF's declared reading order goes unused, and the geometric XY-cut is left to guess at tables it has no need to guess at.

On real documents that guess is wrong. A tagged wind-speed table on a product spec sheet reads column-wise under ColumnAware and drops its header cells entirely:

ColumnAware:  Flagpole Range Product Garden Garden Garden Housebuilder ...
Structure:    Flagpole Range Product Size Style Max Wind Speed - Flagged (mph) ...

Per ISO 32000-1:2008 §14.8.2.3, a pre-order traversal of the structure hierarchy is authoritative for reading order in a Tagged PDF — independent of glyph geometry. The library already computes it (extract_text uses it for tagged docs), but the span API had no way to ask for it.

What this adds

ReadingOrder::Structure. It keeps the geometric order as the baseline and uses the structure tree to fix only the spans it can fix unambiguously — table cells (OrderedContent::in_table): those are reordered in place by their structure rank, so a wide table reads row-major and complete, while every non-table span keeps its geometric position.

Restricting to tables is deliberate, and measured. Applying the tree traversal to the whole page also reorders flowing prose — where the tree's section order can legitimately differ from visual order (it de-prioritises page artifacts, for instance). On a 470-document corpus, whole-page structure order was a wash (88 better / 82 worse against an independent reference); table-scoped was 43 better / 13 worse, and the 13 are pages the reference reader itself extracts empty.

When the tree is absent or not trustworthy for ordering (untagged, or /MarkInfo /Suspects true — already gated by the existing struct_tree_trustworthy), Structure is exactly ColumnAware, so it is always safe to request.

The geometric XY-cut is factored into a private order_spans_column_aware, shared by the ColumnAware and Structure branches (no behaviour change to ColumnAware).

Type of change

New feature (additive enum variant; existing variants unchanged).

Testing performed

tests/structure_reading_order.rs:

• A 2×2 table with far-apart columns whose tags declare row-major order — Structure reads the rows.

• The untagged twin of the same file — Structure falls back to the geometric order byte-for-byte.

• cargo test --features rendering — full suite green (8,862 tests).

• cargo clippy --features rendering -- -D warnings — clean.

• cargo fmt — clean.

Checklist

• Code compiles without warnings
• All tests pass (cargo test)
• Code formatted (cargo fmt)
• Clippy passes (cargo clippy -- -D warnings)
• New code includes tests
• Commit messages follow conventions; DCO signed off
• PR description explains changes

[yfedoseev]

🔴 Changes requested — the feature is correct, but the test doesn't prove it fires

Nice, well-scoped feature: ReadingOrder::Structure reorders only table-cell spans by their structure-tree pre-order rank (row-major for Table→TR→TD/TH) on top of the ColumnAware baseline, leaves non-table spans untouched, and correctly falls back to geometric when the tree is untrusted (/Suspects, untagged) — spec-faithful (§14.8) and non-breaking. No correctness bug in the logic.

Blocker — the marquee test may be vacuous. tests/structure_reading_order.rs:

• structure_falls_back_to_geometry_when_untagged says in a comment: "On this simple 2×2 the geometric order happens to also be row-major."
• If that's true, then structure_order_reads_the_tagged_rows_not_the_columns (asserts [ALPHA, BRAVO, CHARLIE, DELTA]) would pass even if the Structure branch were a no-op, because plain ColumnAware already yields row-major on that fixture. The test never shows Structure differs from ColumnAware.

Please make the tagged test prove the reorder happened: on the same tagged bytes, compute both orders and assert they differ, e.g.

let col = extract_spans_with_reading_order(0, ReadingOrder::ColumnAware);
let str_ = extract_spans_with_reading_order(0, ReadingOrder::Structure);
assert_eq!(col_labels,  ["ALPHA","CHARLIE","BRAVO","DELTA"]); // column-major
assert_eq!(str_labels,  ["ALPHA","BRAVO","CHARLIE","DELTA"]); // row-major via structure
assert_ne!(col_labels, str_labels);

(Widen the column x-gap in the fixture if needed so XY-cut actually goes column-major first.)

Also (non-blocking but worth it):

• Binding reachability — src/python.rs and src/wasm.rs reject "structure" ("Expected 'top_to_bottom' or 'column_aware'"); add Some("structure") => ReadingOrder::Structure so the feature is usable from Python/WASM (per this repo's all-bindings convention).
• Coverage — nested-table, TH-header-row, and multi-span-per-cell cases are explicit design claims but untested; one of each would harden it.

Happy to approve once the tagged test asserts Structure != ColumnAware. Thanks @ajbufort!

[ajbufort]

Thanks for the sharp catch - you're right that the old test was vacuous. Pushed d4748c74 addressing all points.

The blocker (test now proves the reorder fires). XY-Cut deliberately does not split short table cells into columns (per the min_valley_width design note, doing so corrupts cell data), so it can't be coaxed column-major on a clean grid - your suggested ["ALPHA","CHARLIE",...] column-major geometry never materializes. So instead of tuning the gutter, the fixture now draws the rows out of logical sequence (logical row 3 sits highest on the page) - a shape real generators produce. Geometry then follows visual Y (rows 3, 4, 1, 2) while the struct tree declares the true row order:

ColumnAware = [ECHO, FOXTROT, GOLF, HOTEL, ALPHA, BRAVO, CHARLIE, DELTA]  // visual
Structure   = [ALPHA, BRAVO, CHARLIE, DELTA, ECHO, FOXTROT, GOLF, HOTEL]  // tagged

structure_reorders_table_to_logical_order now asserts ColumnAware == GEOMETRIC, Structure == ROW_MAJOR, and assert_ne!(column, structure) - so it can no longer pass with a no-op Structure branch. The untagged test still pins the exact geometric fallback.

Binding reachability. src/python.rs and src/wasm.rs now accept "structure" (both sites each), with updated docstrings and error messages. Verified cargo check --features python and --features wasm.

Coverage. Added one test each for the three design claims:

• structure_keeps_multi_span_cell_contiguous - a two-span cell stays contiguous at its rank;
• structure_orders_th_header_row_first - /TH header cells order like /TD, ahead of a visually-higher data row;
• structure_orders_nested_table_cells_by_preorder - nested-table cells fall in structure pre-order (outer cell, then nested).

Gate: cargo fmt, cargo clippy --tests/--features python/--features wasm all clean; the 5-test suite passes.

[yfedoseev]

Approved — reviewed and regression-passed; in the merge queue. (Formal approval to supersede the earlier review state and record sign-off.)