llm-agent

LLM-related code.

Developing

1. Checkout the code.
2. Create/activate your Python environment of choice.
3. Install uv: pip install uv.
4. Install dependencies: uv pip install -r pyproject.toml.
5. Install dev dependencies: uv pip install -r pyproject.toml --extra dev.
6. Run pre-commit install to install pre-commit hooks.
7. Configure your editor for realtime linting:
   • For VS Code:
     • Set the correct Python environment for the workspace via ctrl+shift+P > Python: Select Interpreter.
     • Install the Pylance and Ruff extensions.
8. Make changes.
9. Verify linting passes scripts/lint.sh.
10. Verify tests pass scripts/test.sh.
11. Commit and push your changes.

Run evaluations

Run pre-defined experiments to evaluate LLM-components and output detailed evaluation reports:

python evals/run_evals.py