Skip to content

Define variants for repeats#1633

Open
douglasjacobsen wants to merge 1 commit into
GoogleCloudPlatform:developfrom
douglasjacobsen:repeat-variants
Open

Define variants for repeats#1633
douglasjacobsen wants to merge 1 commit into
GoogleCloudPlatform:developfrom
douglasjacobsen:repeat-variants

Conversation

@douglasjacobsen

Copy link
Copy Markdown
Collaborator

This merge adds definitions for three new variants related to repeats.

  • is_repeat_parent
  • is_repeat_child
  • repeat_index

These can then be used by objects to conditionally apply definitions based on if the experiment is a repeat or not.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces repeat variants (is_repeat_parent, is_repeat_child, and repeat_index) to Ramble's application base class, registering them as reserved variants and adding a test to verify their presence during analysis. Feedback on these changes highlights an issue where repeat variants are defined without being expanded or typed, and the variant cache is not cleared when the variants dictionary is empty. A suggestion was provided to expand these variables and conditionally clear the cache to prevent stale data.

Comment on lines +724 to +734
# Define variants from repeats
for keyword in [
self.keywords.is_repeat_parent,
self.keywords.is_repeat_child,
self.keywords.repeat_index,
]:
if keyword in variables:
self.object_variants.experiment_variant(
keyword, variables[keyword]
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When defining variants from repeats, the values from variables are not expanded or typed, and the application's variant cache is not cleared if the variants dictionary is empty.

  1. Value Expansion: If any of the repeat variables contain template references or are defined as strings (e.g., "True"/"False"), they should be expanded and typed using self.expander.expand_var(..., typed=True) to ensure consistency with other experiment variants.
  2. Cache Invalidation: If variants is empty, the block if variants: is skipped, meaning self.clear_variant_cache() is never called. This leaves the application with a stale variant cache, which can cause subsequent calls (like self._set_system()) to not see the newly defined repeat variants.

We should expand the values and conditionally clear the variant cache if any repeat variants were defined.

        # Define variants from repeats
        has_repeat_variants = False
        for keyword in [
            self.keywords.is_repeat_parent,
            self.keywords.is_repeat_child,
            self.keywords.repeat_index,
        ]:
            if keyword in variables:
                self.object_variants.experiment_variant(
                    keyword, self.expander.expand_var(variables[keyword], typed=True)
                )
                has_repeat_variants = True

        if has_repeat_variants:
            self.clear_variant_cache()

@ramble-pr-bot

ramble-pr-bot Bot commented Jun 22, 2026

Copy link
Copy Markdown

Ramble Performance Test Metrics

Results produced with commit: 98b7e8b

Test Name Outcome Duration (s) Most Recent Run (s) Last 5 Avg (s)
test_analyze_large_file passed 2.9767 3.0821 (09ce8a0) 3.0300
test_large_template_expansion passed 2.0956 2.1886 (09ce8a0) 2.1971
test_many_experiments passed 33.9816 35.3749 (09ce8a0) 34.7099
test_many_objects_defaults passed 18.4170 19.6031 (09ce8a0) 19.2642
test_matrix_filter_perf passed 1.5653 1.6886 (09ce8a0) 1.6532

@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.19%. Comparing base (09ce8a0) to head (98b7e8b).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1633   +/-   ##
========================================
  Coverage    93.19%   93.19%           
========================================
  Files          345      345           
  Lines        33456    33476   +20     
========================================
+ Hits         31178    31198   +20     
  Misses        2278     2278           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This commit adds definitions for three new variants related to repeats.
 - is_repeat_parent
 - is_repeat_child
 - repeat_index

These can then be used by objects to conditionally apply definitions
based on if the experiment is a repeat or not.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant