Id. citation doesn't capture pin cite with § section references

[lokkju]

Description

When an Id. citation is followed by a section reference using §, the pin cite is not captured. The § is parsed as a separate UnknownCitation (via SectionToken) instead of being included in the Id. citation's pin cite.

Example

From Volokh Amicus Brief (Cato Institute, 2023):

See 42 U.S.C. § 1983. Id. § 394-ccc(2)-(3).

Current behavior

from eyecite import get_citations
cites = get_citations("See 42 U.S.C. § 1983. Id. § 394-ccc(2)-(3).")
for c in cites:
    print(f"{type(c).__name__:25s} matched={c.matched_text():30s} pin_cite={c.metadata.pin_cite}")

Output:

FullLawCitation           matched=42 U.S.C. § 1983               pin_cite=None
IdCitation                matched=Id.                            pin_cite=None
UnknownCitation           matched=§                              pin_cite=None

Expected behavior

FullLawCitation           matched=42 U.S.C. § 1983               pin_cite=None
IdCitation                matched=Id.                            pin_cite=§ 394-ccc(2)-(3)

Root cause

The SectionToken extractor ((\S*§\S*)) matches § before extract_pin_cite() can include it in the Id citation's pin cite. The match_on_tokens() function stops at non-string tokens when strings_only=True, so it never sees the § content.

Other Id. pin cite patterns that DO work

# These all correctly capture pin cites:
"1 U.S. 1. Id. at 5."           # pin_cite="at 5"       ✓
"1 U.S. 1. Id., at 123-24."     # pin_cite="at 123-24"  ✓
"1 U.S. 1. Id. at *5."          # pin_cite="at *5"      ✓
"1 U.S. 1. Id. at 5, 7."        # pin_cite="at 5, 7"    ✓

Possible fixes

1. Extend POST_SHORT_CITATION_REGEX to match § <section> as a pin cite pattern
2. In extract_pin_cite(), don't stop at SectionToken when scanning forward from an Id token
3. Handle § specially in the Id extraction path

Version

eyecite 2.7.6

[mlissner]

Hi @lokkju. We're not currently working on statute parsing with Eyecite, and we're aware that it needs some enhancements. If you're game to publish a PR with this though, we'd welcome that!