fix: distinguish 497K fee tables from footnotes

[sambai-dev]

Problem

A 497K fee-waiver footnote can be rendered as its own one-row table. The Ocean Park filing in #912 includes the phrase "management fees" in that prose, so _classify_table treated the footnote as a second operating-expenses table.

That false positive changed extract_fee_tables to the repeated-section path. The single-column expense parser then read 3 Years as $3, returned no three-year value, and emitted the footnote as a phantom share class. The reduced filing excerpt returned two records with expense_1yr == 3 instead of one record with expenses of $109 and $381.

Approach

• Require fee-label/value structure before classifying a table as operating expenses. Missing values with conventional footnote markers and integer values remain valid, while percent-bearing narrative must match the complete SEC value shape.
• Stop prose-only management-fee tables before they fall through into expense-example, shareholder-fee, performance, or bar-chart classifiers. Mixed tables retain those classifications only when they contain the corresponding structural signal.
• Normalize a shared leading spacer offset across the whole operating-expense or shareholder-fee table so multi-class headers and values remain aligned.
• Preserve malformed nested tables that flatten into one cell only when the label is immediately followed by multiple percentage values, including the existing trailing-empty-cell shape.
• Add a reduced regression fixture preserving the three relevant Ocean Park tables, plus end-to-end cases for marked missing values, percent-bearing prose, single- and multi-class spacer layouts, flattened rows, and indented shareholder fees.
• Update the existing 497K corpus baseline to remove the three phantom records created from one-row prose footnotes.

I rejected filtering malformed classes after extraction because the false table count has already selected the wrong expense-example parser by then. I also rejected a literal prefix check because valid labels can be prefixed or footnote-marked, while prose can itself begin with "management fees."

A corpus-wide classifier comparison changed exactly three tables, all one-row prose footnotes. Existing management-fee label rows remained classified as operating expenses.

Review follow-up

The maintainer-reported fall-through, marked-value, spacer-column, and flattened-row cases are covered directly. Follow-up review also added end-to-end protection for percent-leading narrative, multi-class value alignment, and shareholder-fee extraction after indentation normalization.

Compatibility

This does not change the public API. It narrows an internal classifier false positive and preserves existing SEC value forms, including leading-decimal percentages, split-text percentages, parenthesized values, numeric or symbol markers, and single-letter marker lists.

This intentionally leaves the cover-page ticker fallback from #912 out of scope because it is a separate extraction path with different fallback rules.

Testing

• Every reported and review-discovered behavior was reproduced as an isolated failing test before its source fix.
• uvx hatch run pytest tests/issues/regression/test_issue_912_497k_fee_waiver.py tests/test_prospectus497k.py -q --tb=short: 98 passed.
• Focused 497K characterization command: 50 passed with the same three environment-dependent metadata-only baseline cases excluded; the untouched branch fails those same three cases.
• Fast xdist and coverage gate: 6,433 passed; its 23 failures exactly match the untouched branch.
• Full regression gate: 2,950 passed; its 10 failures exactly match the untouched branch.
• Cassette safety, regression no-skip, regression provenance, Ruff, Pyright, added-line security, and whitespace checks introduced no new findings.
• The exact final diff and its current-base integration passed a fresh fail-closed review with no logic, security, scope, or test-gap findings.

Related to #912

[dgunning]

Automated review of the classifier change. Three findings, all reproduced by running this branch side by side against main on identical HTML.

The narrowing correctly stops the #912 footnote from landing in the operating_expenses bucket, but a footnote that is no longer operating_expenses is not thereby unclassified — it falls through to the later branches, where "first table wins" makes it displace real data. Two of the three findings are that fall-through and the over-tight row predicate that now gates the whole filing.

Non-issues checked: no consumers of the removed _FEE_TABLE_LABELS outside this module; tests/test_497k_tables_characterization.py, tests/issues/regression/test_issue_912_497k_fee_waiver.py and tests/test_prospectus497k.py all pass on this branch (104 passed, 28 deselected); the baseline JSON edits are correct — the only non-phantom change (2016q4-3794) drops three keys whose values were already null.

[sambai-dev]

@dgunning Thanks for the detailed repros. I addressed all three review points:

• Prose-only management-fee tables now stop at classification instead of falling through into expense-example, shareholder-fee, or performance tables. Structured mixed tables still retain their real classification.
• Fee-row detection accepts footnote-marked missing values and integer values, while rejecting percent-bearing narrative that does not have a value shape.
• Leading spacer columns are normalized at the table level so multi-class values stay aligned. The same alignment is applied during shareholder-fee extraction, and the flattened-row fallback now remains reachable when trailing cells are empty.

I added end-to-end regressions for the reported first-table-wins losses, marked missing values, single- and multi-class spacer layouts, flattened rows, and indented shareholder fees.

Local verification on the corrected diff:

• 98 focused issue/prospectus tests passed
• 50 characterization tests passed, with the same three environment-dependent metadata cases excluded from that focused command
• fast: 6,433 passed; the 23 failures exactly match the untouched branch
• regression: 2,950 passed; the 10 failures exactly match the untouched branch
• cassette safety, regression provenance, and diff checks passed