feat: incremental ingestion plus config, activation and anomaly regression tests - #803
Merged
gelluisaac merged 1 commit intoAug 29, 2026
Conversation
…ssion tests Closes Traqora#719, Traqora#718, Traqora#717, Traqora#729. Traqora#729 — incremental ingestion that only touches new ledgers IngestionService.ingest_incremental() asks the state store for the watermark and the network for the head, and walks only what lies between. A cold store starts at the head (backfills stay the job of ingest(start, end)); an unknown or regressed head is a no-op rather than a rewind; max_ledgers bounds a single run and the remainder is picked up next time. Idempotency and restart-safety come from the existing ingest() path, which it delegates to unchanged. Traqora#719 — tests for config validation across config/*.yaml Data-driven sweep of every shipped YAML: parses, non-empty, mapping at the top level, no duplicate keys, no tab indentation, plus enum-drift checks (logging levels, quoted booleans) and the startup dry-run over config/database.yaml with its parsed values pinned. A config added later is covered without anyone remembering to add a test. Fixes a bug this uncovered: ValidationResult.valid had no default, so every validator in config_dry_run raised TypeError on construction and the dry-run path was unrunnable. Traqora#718 — model version activation and rollback ModelRegistry.activate() and rollback_to_version() now share one _switch_serving_version() that demotes the outgoing version, promotes the incoming one and records lineage on both in a single commit, so a failure mid-switch cannot leave the model unserved. Lineage was previously assigned to ModelVersion.metadata — SQLAlchemy's reserved MetaData attribute, not a column — so it was silently discarded. Adds a real lineage JSON/JSONB column and migration 011. The torch import in mlflow_tracker is now lazy so astroml.tracking is importable without torch installed. Traqora#717 — anomaly detection and calibration regression tests Pins detector output on seeded synthetic injected-fraud data (only the injected accounts are flagged; a clean ledger flags nothing) and the Platt/isotonic calibration contract: Brier score improves, output stays in bounds, the transform stays monotonic, refits are stable, and degenerate input does not produce NaN. Existing suite is unchanged: 123 failures / 82 collection errors before and after, all from optional dependencies absent in this environment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@ritik4ever Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
📏 Pull request size warningThis pull request exceeds the review-friendly size limits. CI is not blocked — this is guidance, not a gate.
Why it matters Review quality drops sharply with diff size: defects slip through, review turnaround grows, and reverts become riskier because unrelated changes are bundled together. What to do
Legitimately large?
Breached limits
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #719
Closes #718
Closes #717
Closes #729
#729 — IngestionService.ingest_incremental(): reads the watermark, asks for the head, touches only what's new. Cold store starts at head; unknown/regressed head is a no-op; max_ledgers bounds a run. Delegates to ingest() so idempotency and restart-safety are unchanged.
#719 — Data-driven sweep over every shipped YAML (parses, mapping, no duplicate keys, no tabs, enum drift) + startup dry-run on config/database.yaml with parsed values pinned. Fixes a bug: ValidationResult.valid had no default, so every validator raised TypeError and the dry-run path was unrunnable.
#718 — activate() and rollback_to_version() now share one switch that demotes, promotes and records lineage in a single commit, so a mid-switch failure can't leave the model unserved. Fixes a bug: lineage was assigned to ModelVersion.metadata (SQLAlchemy's reserved attribute, not a column) and silently discarded — adds a real lineage column + migration 011. Torch import made lazy so astroml.tracking imports without torch.
#717 — Pins detector output on seeded synthetic injected-fraud data and the Platt/isotonic calibration contract (Brier improves, in-bounds, monotonic, stable refits, no NaN on degenerate input).
228 new tests pass. Existing suite unchanged: 123 failures / 82 collection errors before and after, all from optional deps missing locally.