Skip to content

introduce iterator helpers that manage recursion / time#454

Merged
davidhewitt merged 12 commits into
mainfrom
dh/iteration-pattern
May 29, 2026
Merged

introduce iterator helpers that manage recursion / time#454
davidhewitt merged 12 commits into
mainfrom
dh/iteration-pattern

Conversation

@davidhewitt
Copy link
Copy Markdown
Collaborator

@davidhewitt davidhewitt commented May 19, 2026

This generally seeks to avoid cases where recursion inside iteration could lead to resource overflows. I introduce iteration helpers to each container which check both recursion and time on each turn.


Summary by cubic

Introduces recursion/time-aware lending iterators for core containers and rebuilds json.dumps to stream with them. Deep nesting now raises RecursionError, JSON cycles are detected, comparison/sort errors propagate, and resource exhaustion never corrupts heap state.

  • Refactors

    • Added lending iterators for lists, tuples, named tuples, dicts, and sets that hold a recursion token and call check_time() on each step.
    • Rebuilt json.dumps around a streaming encoder using these iterators; dicts pre-collect entries for skipkeys/sort_keys, take an explicit recursion token, and track active containers to detect cycles.
    • Updated py_eq/py_cmp across core types and Value to return RunResult; sorting passes errors through without extra wrapping.
    • Added a test-only sys.setrecursionlimit() (behind test-hooks), backed by ResourceTracker::lower_recursion_limit and forwarded via monty-python.
  • Bug Fixes

    • list.count/index/remove and dict.update iterate under recursion tokens; cycles don’t panic; deep comparisons raise RecursionError.
    • Dict/set iteration detects size changes and raises a runtime error.
    • enumerate and bytes.splitlines build results under HeapGuard so early exits/timeouts don’t leave partial heap state.
    • Deserialization of LimitedTracker is now backward-compatible with prior formats.
    • Tests cover deep JSON recursion, cycle detection, and runtime limit lowering; the datatest harness uses the sys.setrecursionlimit() hook.

Written for commit 5971c5c. Summary will update on new commits. Review in cubic

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 19, 2026

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 15 skipped benchmarks1


Comparing dh/iteration-pattern (172372e) with main (45a3b2d)

Open in CodSpeed

Footnotes

  1. 15 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 19, 2026

Codecov Results 📊

❌ Patch coverage is 10.00%. Project has 24165 uncovered lines.
❌ Project coverage is 66.15%. Comparing base (base) to head (head).

Files with missing lines (25)
File Patch % Lines
crates/monty/src/modules/json/dump.rs 0.00% ⚠️ 146 Missing
crates/monty/src/types/dict.rs 0.00% ⚠️ 74 Missing
crates/monty/src/types/list.rs 0.00% ⚠️ 60 Missing
crates/monty/src/types/namedtuple.rs 0.00% ⚠️ 53 Missing
crates/monty/src/types/dict_view.rs 0.00% ⚠️ 46 Missing
crates/monty/src/types/set.rs 0.00% ⚠️ 43 Missing
crates/monty/src/types/tuple.rs 46.88% ⚠️ 34 Missing and 4 partials
crates/monty/src/types/bytes.rs 0.00% ⚠️ 6 Missing
crates/monty/src/builtins/enumerate.rs 0.00% ⚠️ 4 Missing
crates/monty/src/types/date.rs 0.00% ⚠️ 2 Missing
crates/monty/src/types/datetime.rs 0.00% ⚠️ 2 Missing
crates/monty/src/types/str.rs 0.00% ⚠️ 2 Missing
crates/monty/src/types/timedelta.rs 0.00% ⚠️ 2 Missing
crates/monty/src/sorting.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/dataclass.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/file.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/path.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/py_trait.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/range.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/re_match.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/re_pattern.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/slice.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/test_cm.rs 0.00% ⚠️ 1 Missing
crates/monty/src/types/timezone.rs 0.00% ⚠️ 1 Missing
crates/monty/src/value.rs 66.67% ⚠️ 1 Missing
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
- Coverage    66.19%    66.15%    -0.04%
==========================================
  Files          279       279         —
  Lines        71052     71389      +337
  Branches    152853    153482      +629
==========================================
+ Hits         47030     47224      +194
- Misses       24022     24165      +143
- Partials      3993      4036       +43

Generated by Codecov Action

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 25 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="crates/monty/src/types/dict_view.rs">

<violation number="1" location="crates/monty/src/types/dict_view.rs:70">
P1: These view equality helpers still squash membership errors into `false`, so cyclic key/item comparisons can return unequal instead of raising `RecursionError`.</violation>
</file>

<file name="crates/monty/src/types/set.rs">

<violation number="1" location="crates/monty/src/types/set.rs:285">
P1: Propagate `other.contains` errors here instead of collapsing them into `false`.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread crates/monty/src/types/dict_view.rs
Comment thread crates/monty/src/types/set.rs
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread crates/monty/src/modules/json/dump.rs
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 13 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread crates/monty/src/resource.rs Outdated
@davidhewitt davidhewitt merged commit 0039ec3 into main May 29, 2026
30 checks passed
@davidhewitt davidhewitt deleted the dh/iteration-pattern branch May 29, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant