Skip to content

Variance/jitter cannot be disabled for automatic retry/random exponential backoff #2026

@RealErikHuebner

Description

@RealErikHuebner

Bug description

When configuring automatic retry with random exponential backoff for genai.Client objects, disabling the variance/jitter of retry wait times is impossible using genai.types.HttpRetryOptions objects; setting the variance/jitter to 0.0 automatically defaults to 1.0 instead upon initialization of genai.Client objects.

The issue occurs in google/genai/_api_client.py in retry_args():

jitter=options.jitter or _RETRY_JITTER,

The falsy check returns False when 0.0 is passed. Since jitter is typed and validated as Float | None, this line should instead be:

jitter=options.jitter if options.jitter is not None else _RETRY_JITTER,

or:

jitter=_RETRY_JITTER if options.jitter is None else options.jitter,

Expected behaviour

Initializing a genai.Client object with the http_options.retry_options parameter set to:

genai.types.HttpRetryOptions(jitter = 0.0)

should return a genai.Client object that has no variance/jitter added to retry wait times.

Actual behaviour

Initializing a genai.Client object with the http_options.retry_options parameter set to:

genai.types.HttpRetryOptions(jitter = 0.0)

returns a genai.Client object that is initialized to add a random value between 0 seconds and 1 second to retry wait times.

Steps to reproduce

  1. Run the following code:
from google import genai

client = genai.Client(
    vertexai=True,
    project="foo",
    location="foo",
    http_options=genai.types.HttpOptions(
        api_version="v1",
        retry_options=genai.types.HttpRetryOptions(
            jitter=0.0,
        ),
    ),
)

print(client._api_client._retry.wait.jitter)

The above block will print:

1

Environment details

  • Programming language: Python
  • OS: Any

Metadata

Metadata

Assignees

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions