Repository navigation
Opt-in keyless You.com search provider, as a second keyless option next to DuckDuckGo #629
brainsparker
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I work at You.com, so treat the recommendation accordingly. There's no CONTRIBUTING file, so I'm asking here before writing any code.
Context. #179 counted 1,341
Exa returned HTTP 429errors in a GAIA validation campaign, 44% of the 3,039 tool failures, with the DuckDuckGo fallback quietly serving the rest. Since then #199 added same-provider retry and the startup notice, #241 parks a rate-limited provider, #274 persists the cache and cooldown, and #609 (ATO-120) takes keyless Exa out of the chain. So the 429 loop itself is handled.What's left, as I read
search-orchestrator.tsandweb-config.ts: with no keys and no SearXNG instance, DuckDuckGo's HTML endpoint is the only provider that can answer. #179 called it "materially weaker for the factual lookups this workload needs", and #237 mentions the challenge page.Idea. A
youprovider insrc/tools/os/web-search/providers/, opt-in and off by default.GET https://api.you.com/v1/agents/search?query=...&count=..., JSON results, no account. The free tier is 100 searches a day. Past that it answers 429 like any quota, so it goes throughassertProviderStatusand the existing cooldown. To be clear, this doesn't make rate limits go away. It adds one more keyless provider, with a documented quota, that a user can put in front of DuckDuckGo.YDC_API_KEY:https://ydc-index.io/v1/searchwith anX-API-Keyheader, for more volume. A campaign at Keyless Exa search returns HTTP 429 continuously and degrades to a weaker provider silently #179's scale needs a key. The free tier won't cover it.provider: "you", or"you"infallback. The default config doesn't change. The docs say plainly that queries go to a hosted third-party API.WebSearchProviderNameunion andparseWebSearchProviderName, a case inprovider-registry.tsandisProviderUsable(usable without a key; the key only switches the endpoint), aweb.search.youconfig block (endpoint,apiEndpoint,apiKeyEnv: "YDC_API_KEY"), ayou-provider.test.tslike the Brave one, and the README provider list.For transparency:
searchHttpsends a fixed Chrome User-Agent on every search request. On You.com requests only, I'd sendUser-Agent: youdotcom-integration/atomicbot-ai-atomic-agentinstead, so we can see traffic that comes from Atomic Agent. That needs a small change so a provider-suppliedUser-Agentheader replaces the default rather than adding a second one. Nothing changes for the other providers.Measuring it. If it'd help you decide, I can run a slice of your GAIA validation setup with
youas the primary (keyed, so the free quota doesn't skew it) next to your current config, and post the tool-failure breakdown in the same shape as #179. Tell me which split and settings you'd want. I'm not promising numbers up front.If you'd rather not add another hosted provider right now, say so and I'll close this without fuss.
All reactions