This is a?
New feature
Detailed description
Summary
Add a valkey_fixwin rate-limiting algorithm that provides distributed, cross-worker fixed-window counting backed by Valkey. It would register through the existing @algo.algo_class registry alongside the current in-memory fixwin algorithm (which stays the default) and implement the existing ReteLimitAlgo ABC (require_access / release_access) — no new interface. Cross-worker counting is done with atomic INCR + EXPIRE via a small Lua script.
Motivation
LangBot's current rate limiter (pipeline/ratelimit/algos/fixedwin.py) keeps its counters in process memory (dict + asyncio.Lock). That works for a single-process deployment, but when LangBot is scaled horizontally to multiple workers, each process keeps its own independent counters. The effective limit becomes configured_limit × number_of_workers, so the configured cap is not enforced correctly across the deployment.
Operators running LangBot in toB/SaaS or otherwise scaled setups need limits enforced consistently regardless of which worker handles a message. A shared counter store solves this directly, and Valkey — the open-source, Linux Foundation–backed fork of Redis — is a natural fit since many such deployments already run it. Making the algorithm pluggable and opt-in keeps single-process users completely unaffected.
Proposed Implementation
- New
src/langbot/pkg/pipeline/ratelimit/algos/valkey_fixwin.py with a ValkeyFixedWindowAlgo class decorated @algo.algo_class('valkey_fixwin'), implementing the existing ReteLimitAlgo ABC and mirroring the window semantics of the in-memory fixwin.
- Atomic cross-worker fixed-window counting: per
(launcher_type, launcher_id, window_start) key, a single Lua script does check-then-INCR and sets EXPIRE on first increment, so the window reaps itself via TTL and the increment is atomic across workers.
- Make the algorithm config-selectable via
safety.rate-limit.algo. Today pipeline/ratelimit/ratelimit.py hardcodes algo_name = 'fixwin'; this would read the algo from config and default to 'fixwin' (backward-compatible). Add the selector to the safety.yaml pipeline metadata (with en_US + zh_Hans labels) and an "algo": "fixwin" default to the default pipeline config.
- A top-level
valkey: config block (host/port/db/username/password/tls) for the shared connection. Reuses the valkey-glide client convention (client_name, lazy_connect=True, optional auth/TLS).
- Opt-in / disabled by default:
fixwin (in-memory) remains the default; valkey_fixwin is inert unless explicitly selected.
- Fail-open: if Valkey is unreachable, the limiter logs a throttled warning and allows the request rather than dropping legitimate traffic — a rate limiter is a safety throttle, not an auth gate. (A fail-closed toggle could be added later.)
Dependencies
valkey-glide>=2.4.1,<3.0.0 (optional dependency; official async-native Valkey client with reconnect/backoff, TLS, lazy-connect). License is permissive (Apache-2.0 / BSD).- A Valkey server for deployments that opt into the distributed algorithm (e.g. the
valkey/valkey-bundle:9.1.0 image).
Testing Plan
- Unit tests (mocked glide client, no server): key schema + window math, allow within limit, deny over limit (drop), wait-strategy retry into the next window, fail-open on client error, client-config build (
client_name / lazy_connect / auth / TLS), and registry membership. The existing in-memory fixwin unit tests are retained as a regression guard, plus algo-selection tests (default stays fixwin, unknown algo raises).
- Integration tests against a live
valkey/valkey-bundle:9.1.0 instance, including a multi-worker shared-counter test: two separate algorithm instances with their own clients (simulating two worker processes) point at the same Valkey key; together they may only consume the configured limit, and one more request from either instance is denied — proving cross-process limiting. These follow the existing slow-gated / env-gated test precedent (same pattern as the Postgres migration tests).
I'd be happy to contribute this as a PR if it aligns with the project roadmap. It's purely additive, keeps the in-memory fixwin as the default, and reuses the existing algorithm registry and ReteLimitAlgo ABC. The one cross-cutting change to flag is making ratelimit.py read the algorithm from config instead of hardcoding 'fixwin' (backward-compatible via the default).
This is a?
New feature
Detailed description
Summary
Add a
valkey_fixwinrate-limiting algorithm that provides distributed, cross-worker fixed-window counting backed by Valkey. It would register through the existing@algo.algo_classregistry alongside the current in-memoryfixwinalgorithm (which stays the default) and implement the existingReteLimitAlgoABC (require_access/release_access) — no new interface. Cross-worker counting is done with atomicINCR+EXPIREvia a small Lua script.Motivation
LangBot's current rate limiter (
pipeline/ratelimit/algos/fixedwin.py) keeps its counters in process memory (dict+asyncio.Lock). That works for a single-process deployment, but when LangBot is scaled horizontally to multiple workers, each process keeps its own independent counters. The effective limit becomesconfigured_limit × number_of_workers, so the configured cap is not enforced correctly across the deployment.Operators running LangBot in toB/SaaS or otherwise scaled setups need limits enforced consistently regardless of which worker handles a message. A shared counter store solves this directly, and Valkey — the open-source, Linux Foundation–backed fork of Redis — is a natural fit since many such deployments already run it. Making the algorithm pluggable and opt-in keeps single-process users completely unaffected.
Proposed Implementation
src/langbot/pkg/pipeline/ratelimit/algos/valkey_fixwin.pywith aValkeyFixedWindowAlgoclass decorated@algo.algo_class('valkey_fixwin'), implementing the existingReteLimitAlgoABC and mirroring the window semantics of the in-memoryfixwin.(launcher_type, launcher_id, window_start)key, a single Lua script does check-then-INCRand setsEXPIREon first increment, so the window reaps itself via TTL and the increment is atomic across workers.safety.rate-limit.algo. Todaypipeline/ratelimit/ratelimit.pyhardcodesalgo_name = 'fixwin'; this would read the algo from config and default to'fixwin'(backward-compatible). Add the selector to thesafety.yamlpipeline metadata (with en_US + zh_Hans labels) and an"algo": "fixwin"default to the default pipeline config.valkey:config block (host/port/db/username/password/tls) for the shared connection. Reuses the valkey-glide client convention (client_name,lazy_connect=True, optional auth/TLS).fixwin(in-memory) remains the default;valkey_fixwinis inert unless explicitly selected.Dependencies
valkey-glide>=2.4.1,<3.0.0(optional dependency; official async-native Valkey client with reconnect/backoff, TLS, lazy-connect). License is permissive (Apache-2.0 / BSD).valkey/valkey-bundle:9.1.0image).Testing Plan
client_name/lazy_connect/ auth / TLS), and registry membership. The existing in-memoryfixwinunit tests are retained as a regression guard, plus algo-selection tests (default staysfixwin, unknown algo raises).valkey/valkey-bundle:9.1.0instance, including a multi-worker shared-counter test: two separate algorithm instances with their own clients (simulating two worker processes) point at the same Valkey key; together they may only consume the configured limit, and one more request from either instance is denied — proving cross-process limiting. These follow the existing slow-gated / env-gated test precedent (same pattern as the Postgres migration tests).I'd be happy to contribute this as a PR if it aligns with the project roadmap. It's purely additive, keeps the in-memory
fixwinas the default, and reuses the existing algorithm registry andReteLimitAlgoABC. The one cross-cutting change to flag is makingratelimit.pyread the algorithm from config instead of hardcoding'fixwin'(backward-compatible via the default).