Skip to content

fix(engine): clamp extreme historical dates to valid Python datetime range - #3403

Closed
handnewb wants to merge 1 commit into
vectorize-io:mainfrom
handnewb:fix/safe-date-clamping-extreme-years
Closed

fix(engine): clamp extreme historical dates to valid Python datetime range#3403
handnewb wants to merge 1 commit into
vectorize-io:mainfrom
handnewb:fix/safe-date-clamping-extreme-years

Conversation

@handnewb

Copy link
Copy Markdown
Contributor

Summary

Fixes #3217: Stored facts can carry BC-era dates (e.g., year -534) or far-future projections that survive database ingestion but crash datetime.fromisoformat() with ValueError: year N is out of range. This poisons the entire recall/consolidation path for the affected bank — the failure is deterministic and retries don't help.

Root Cause

datetime.fromisoformat() accepts only years 1–9999. Dates outside this range pass through PostgreSQL storage (which accepts any ISO 8601 string) but crash when the consolidation engine or memory-edit path tries to parse them.

Changes

consolidator.py — consolidation date comparisons

New _safe_fromisoformat() wrapper used by _as_dt():

  • Years < 1 → clamp to datetime(1, 1, 1)
  • Years > 9999 → clamp to datetime(9999, 12, 31, 23, 59, 59)
  • Extracts year via regex from the error message to decide direction
  • Preserves the temporal signal (very old / very new) while preventing a crash

memory_engine.py — memory-unit edit path

Same clamping logic added to _parse_edit_date() with an inline try/except + regex fallback.

Impact

  • Banks with ancient history facts no longer crash the entire recall pipeline
  • Consolidation runs complete instead of retrying forever
  • Direction-preserving: BC dates → year 1, far-future → year 9999

@handnewb
handnewb force-pushed the fix/safe-date-clamping-extreme-years branch from 552ceb1 to a17ff1e Compare August 11, 2026 21:24
…range

Stored facts can carry BC-era dates or far-future projections that survive
database ingestion but crash datetime.fromisoformat() with 'year N is out
of range'.  This poisons the entire recall / consolidation path for the
affected bank, since the failure is deterministic.

Changes:
- consolidator.py: new _safe_fromisoformat() wrapper used by _as_dt() in
  the consolidation date-comparison path.  Years < 1 clamp to year 1;
  years > 9999 clamp to year 9999, preserving the temporal signal (very
  old / very new) while preventing a crash.
- memory_engine.py: same clamping logic added to _parse_edit_date() for
  the memory-unit edit path.

Closes vectorize-io#3217.
@nicoloboschi

Copy link
Copy Markdown
Collaborator

Closing in favor of #3413, which fixes the actual mechanism. The crash isn't fromisoformat on stored strings: fromisoformat raises "Invalid isoformat string" (not "year N is out of range") for negative-year input, _as_dt runs after recall returns so its errors can't produce the observed "Failed to search memories (...)" wrapper, and no out-of-range date can reach the DB through asyncpg in the first place. The observed years (-534/-974/-97974) are exactly now.year − {2560, 3000, 100000} — query-time "N years ago" arithmetic in extract_period, fixed at that layer in #3413. The naive datetime(1,1,1) clamp here would also introduce naive-vs-aware TypeErrors in _merge_min/_merge_max. Thanks for taking a look at this one regardless!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search: extreme historical dates in stored facts crash recall with 'year N is out of range'

2 participants