[codex] Add LLM observability guards#435
Open
Dreamstore2046 wants to merge 1 commit into
Open
Conversation
|
@Dreamstore2046 is attempting to deploy a commit to the Harrison Eze's projects Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
event,model,latency_ms,requestId, and serializederrorconsole.*use fromagent/and add an agent-specific ESLint no-console guardagent_llm_error_total / agent_llm_calls_total > 5%over 10 minutesValidation
npm test -- shared/__tests__/metrics.test.ts agent/__tests__/llm-error-fallback.test.ts agent/__tests__/no-console.test.tsnpm run lint:agentnpx tsc --noEmit --target ES2022 --module ESNext --moduleResolution bundler --allowImportingTsExtensions --esModuleInterop --allowSyntheticDefaultImports --strict --skipLibCheck --types node,vitest shared/metrics.ts shared/__tests__/metrics.test.tsnode -e "const fs=require('fs'); const yaml=require('js-yaml'); for (const f of ['docker/prometheus/prometheus.yml','docker/prometheus/alerts.yml']) { yaml.load(fs.readFileSync(f,'utf8')); } console.log('prometheus yaml ok')"git diff --cached --checkNotes:
tscincludingagent/server.tsis still blocked by existing unrelated project errors inagent/tools.tsx402 template-literal arguments andshared/rate-limit.tstypings.docker compose config --quietcould not run in this local environment because Docker is not installed.Closes #243.