Skip to content

Preserve saved globals when copying DelayEvaluator - #3346

Open
Hashim1999164 wants to merge 2 commits into
Netflix:masterfrom
Hashim1999164:fix/delay-evaluator-copy-globals
Open

Preserve saved globals when copying DelayEvaluator#3346
Hashim1999164 wants to merge 2 commits into
Netflix:masterfrom
Hashim1999164:fix/delay-evaluator-copy-globals

Conversation

@Hashim1999164

@Hashim1999164 Hashim1999164 commented Aug 22, 2026

Copy link
Copy Markdown

PR Type

[x] Bug fix
[ ] New feature
[ ] Core Runtime change (higher bar; see CONTRIBUTING.md)
[ ] Docs / tooling
[ ] Refactoring

Summary

Attribute and item access on DelayEvaluator copy the object and previously dropped the caller globals captured by config_expr. Custom functions then raised NameError after any chained access.

Issue

Fixes #3345

Reproduction

Runtime: local

Commands to run:

PYTHONPATH=. pytest test/unit/test_delay_evaluator.py

Where evidence shows up: unit test output

Before (error / log snippet)
After copy or attribute access, DelayEvaluator._globals became None, so eval of config_expr("my_func()") lost my_func and raised NameError.
After (evidence that fix works)
test_copy_preserves_saved_globals PASSED
test_deepcopy_preserves_saved_globals PASSED
test_getattr_preserves_saved_globals PASSED
test_getitem_preserves_saved_globals PASSED
test_chained_call_uses_saved_globals PASSED
test_copied_call_uses_saved_globals PASSED

Root Cause

copy and deepcopy constructed DelayEvaluator(self._config_expr) without saved_globals. getattr and getitem call copy, so chaining dropped the globals config_expr had stored.

Why This Fix Is Correct

Pass saved_globals=self._globals into both copy paths so the reference is kept, matching the previous comment that globals stay as a reference. Scope is limited to those two methods plus unit tests.

Failure Modes Considered

  1. Deep copy still keeps globals by reference (intentional; same object identity as before for globals).
  2. Evaluators created without saved_globals still copy None correctly.

Tests

[x] Unit tests added/updated
[ ] Reproduction script provided (required for Core Runtime)
[ ] CI passes
[ ] If tests are impractical: explain why below and provide manual evidence above

Non Goals

No change to eval semantics beyond preserving the globals reference across copy.

AI Tool Usage

[x] No AI tools were used in this contribution
[ ] AI tools were used (describe below)

Attribute and item access copy the evaluator and previously dropped caller globals, so config_expr with custom functions failed after chaining.
@greptile-apps

greptile-apps Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR preserves caller globals by reference when copying or deep-copying a DelayEvaluator, fixing chained attribute and item evaluation of custom expressions.

  • Passes saved_globals through both copy implementations.
  • Adds regression tests for copying, deep-copying, attribute and item access, and result-level chained evaluation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously requested chained result-level evaluation is now covered and would fail under the original globals-dropping behavior.

Important Files Changed

Filename Overview
metaflow/user_configs/config_parameters.py Copy and deep-copy operations now retain the saved globals reference while preserving the existing access-copy behavior.
test/unit/test_delay_evaluator.py Adds focused regression coverage, including a chained call that reaches evaluation and asserts the resolved result.

Reviews (2): Last reviewed commit: "Cover chained DelayEvaluator evaluation ..." | Re-trigger Greptile

Comment thread test/unit/test_delay_evaluator.py

@Shriprasad-P Shriprasad-P left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I independently reproduced the namespace-loss issue and verified that the implementation fixes it.

Preserving saved_globals by reference in both __copy__ and __deepcopy__ matches the existing DelayEvaluator semantics. Chained attribute/item evaluation that fails with a NameError on the base revision succeeds on this PR, while the copied access state remains independent.

One non-blocking coverage gap remains: the new tests verify _globals / _access directly but never call the copied or chained DelayEvaluator. Adding one regression test that evaluates the chained expression and asserts its returned value would cover the actual user-visible failure path rather than only the internal state.

Small comment nit as well: the new __copy__ comment uses config_expr("my_func()") as the example for attribute/item access, but non-identifier expressions have _access=None and don't take that chaining path. An identifier-based example such as config_expr("my_func").project would describe the affected path more accurately.

The implementation itself looks correct.

@Hashim1999164

Copy link
Copy Markdown
Author

@Shriprasad-P yeah fair those tests were only poking at _globals. added one that actually calls the chained evaluator and checks the return value. also swapped the comment to config_expr("my_func").project, my_func() was the wrong example there

@SahilKumar75

Copy link
Copy Markdown

Thanks for preparing the implementation. I originally reported this issue with the same fix and had a local patch and tests ready, but I was waiting for maintainer acknowledgement as required by the contribution guidelines. I am happy to help review or test this change if useful.

@Hashim1999164

Copy link
Copy Markdown
Author

@SahilKumar75 ah i didnt know you already had a patch sitting around. this was just copy dropping saved_globals. if you wanna poke at the tests go for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DelayEvaluator.__copy__/__deepcopy__ drop saved_globals, breaking config_expr() with custom functions

3 participants