Skip to content

Commit a082f46

Browse files
authored
Refactor Claude skills (#348)
1 parent b3c1d40 commit a082f46

16 files changed

Lines changed: 603 additions & 188 deletions

‎.claude/CLAUDE.md‎

Lines changed: 10 additions & 155 deletions
Original file line numberDiff line numberDiff line change
@@ -17,164 +17,19 @@ Tenant First Aid is a chatbot for Oregon housing/eviction legal information. Fla
1717
- **Frontend** (`frontend/`): React 19 + TypeScript + Vite + Tailwind CSS 4. Uses `@langchain/core` message types (`HumanMessage`/`AIMessage`) directly for chat state. Streaming via native `ReadableStream`.
1818
- **Type bridge**: Frontend TypeScript types in `src/types/` are auto-generated from backend Pydantic models via `generate-types` and gitignored. Must regenerate before building or type-checking.
1919

20-
## Backend workflow (run from `backend/`)
21-
22-
```bash
23-
make fmt # Format + sort imports (ruff)
24-
make lint # Lint (ruff)
25-
make typecheck # Type-check (ty)
26-
make test # Run tests (pytest)
27-
make --keep-going check # All of the above in one shot
28-
29-
# Single test
30-
uv run pytest -s -k <test_name>
31-
32-
# LangChain-specific tests
33-
uv run pytest -m langchain
34-
35-
# Coverage
36-
make test TEST_OPTIONS="--cov tenantfirstaid --cov-report html --cov-branch"
37-
```
38-
39-
All python commands should be run via `uv run python ...`
40-
41-
### Environment variables
42-
43-
See `backend/.env.example`. Key ones: `MODEL_NAME`, `GOOGLE_APPLICATION_CREDENTIALS`, `VERTEX_AI_DATASTORE_LAWS`, `LANGSMITH_API_KEY`.
44-
45-
### LangSmith evaluations
46-
47-
```bash
48-
# Run LangChain-specific tests
49-
uv run pytest -m langchain
50-
51-
# Run with LangSmith tracing (requires API key)
52-
LANGSMITH_TRACING=true LANGCHAIN_TRACING_V2=true uv run pytest -m langchain
53-
54-
# Run evaluations (see backend/evaluate/EVALUATION.md)
55-
uv run python -m evaluate.run_langsmith_evaluation --num-repetitions 20
56-
```
57-
58-
See `docs/EVALUATION.md` for details.
59-
60-
## Frontend workflow (run from `frontend/`)
61-
62-
```bash
63-
npm run generate-types # Required before build/typecheck — generates src/types/models.ts
64-
npm run lint # Lint (eslint)
65-
npm run format # Format (prettier)
66-
npm run typecheck # Type-check (tsc)
67-
npm run build # Build (auto-generates types first)
68-
npm run test -- --run # Run tests (vitest)
69-
npm run test -- --run --coverage # With coverage
70-
```
71-
72-
`generate-types` requires `uv` to be installed (it runs backend Python to emit JSON Schema, piped through `json2ts`).
73-
7420
## Style notes
7521

7622
- Write comments as full sentences and end them with a period.
7723

78-
## Document files (`backend/scripts/documents/`)
79-
80-
All `.txt` law documents must be **pure ASCII**. Vertex AI RAG ingestion has produced mojibake when UTF-8 is present. Before adding or modifying any `.txt` file in this directory, verify with:
81-
82-
```bash
83-
LC_ALL=C grep -n '[^[:print:][:space:]]' path/to/file.txt # must produce no output
84-
```
85-
86-
Common offenders and their ASCII replacements (see `ASCII_REPLACEMENTS` in `backend/scripts/generate_metadata_jsonl.py` for the full list):
87-
88-
| Character | Unicode | Replace with |
89-
|---|---|---|
90-
| `'` | U+2019 right single quote | `'` |
91-
| `"` `"` | U+201C/U+201D double quotes | `"` |
92-
| `§` | U+00A7 section sign | `Section ` |
93-
| `§§` | U+00A7 U+00A7 | `Sections ` |
94-
| `—` | U+2014 em dash | `--` |
95-
| `–` | U+2013 en dash | `-` |
96-
| `•` | U+2022 bullet | `-` |
97-
98-
`make generate-metadata` auto-converts all known offenders in place and warns with a suggested replacement for any it cannot handle.
99-
100-
### Publication cadence
101-
102-
Oregon publishes statutes and annotations on offset biennial cycles, which is why statutes and annotations live in different year subfolders:
103-
104-
- **Statutes** (`ORS*.txt`, `OAR*.txt`) — published in odd years after each long legislative session. See the [current edition](https://www.oregonlegislature.gov/bills_laws/pages/ors.aspx).
105-
- **Annotations** (`ORS*_annotations.txt`) — published in the fall of each even-numbered year as a Cumulative Supplement. See the [current edition](https://www.oregonlegislature.gov/bills_laws/Pages/Annotations.aspx).
106-
107-
Statute and annotation chapter pages are accessible at predictable URLs that aren't directly enumerated from the landing pages (which route through PDFs and SharePoint volume groupings):
108-
109-
- Statutes: `https://www.oregonlegislature.gov/bills_laws/ors/orsNNN.html`
110-
- Annotations: `https://www.oregonlegislature.gov/bills_laws/ors/anoNNN.html`
111-
112-
`NNN` is the three-digit zero-padded chapter number, with lowercase letter suffix for sub-chapters. Examples: chapter 90 → `ors090.html` / `ano090.html`; chapter 659A → `ors659a.html` / `ano659a.html`.
113-
114-
## Commit messages
115-
116-
Concise, imperative mood, small focused commits. Write like a humble experienced engineer — casual, no listicles, highlight non-obvious choices. No robot speak, marketing buzzwords, or vague fluff.
117-
118-
## Pull Request expectations
119-
120-
Use the PR template. For frontend, backend, and backend/scripts changes:
121-
- Add tests when needed
122-
- Update documentation
123-
- Run `make lint` and `make fmt`
124-
- Full test suite passes
125-
126-
### Pull Request reviews
127-
128-
Perform a comprehensive code review of the modified code with the following focus areas:
129-
130-
1. **Code Quality**
131-
- Consistent style: formatted with `ruff format`, imports sorted, type hints passing `make typecheck`
132-
- Clean code principles and best practices
133-
- Suggest improvements to modularity, composition and dependency inversion where appropriate
134-
- Identify opportunities for refactoring or simplification
135-
- Check for code duplication and suggest DRY improvements
136-
- Identify and suggest removal of dead code or redundant logic
137-
- Proper error handling and edge cases
138-
- Code readability and maintainability, including type hints where appropriate
139-
- Prefer strongly typed interfaces and data structures over stringly-typed where possible
140-
- Consistent environment variable declarations across GitHub Actions, `.env.example`, tests, and docs — no orphaned secrets/variables
141-
2. **Security**
142-
- Check for potential security vulnerabilities
143-
- Validate input sanitization
144-
- Review authentication/authorization logic
145-
- Ensure secrets are handled securely and not exposed in code or logs
146-
- Flag obsolete secrets/configuration/environment variables
147-
3. **Performance**
148-
- Identify potential performance bottlenecks
149-
- Review database queries for efficiency
150-
- Check for memory leaks or resource issues
151-
4. **Testing**
152-
- Verify adequate test coverage
153-
- Tests covering new behaviour
154-
- Review test quality and edge cases
155-
- Check for missing test scenarios
156-
5. **Documentation**
157-
- Ensure code is properly documented
158-
- Clear documentation for public API changes
159-
- Verify README updates for new features
160-
- Check API documentation accuracy
161-
- Clean history and helpful PR description
162-
163-
Provide detailed feedback using inline comments for specific issues.
164-
Use top-level comments for general observations or praise.
165-
166-
167-
168-
## GitHub Actions Security
169-
170-
We pin all third-party actions to commit SHAs to prevent supply chain attacks:
171-
172-
```yaml
173-
# Good: SHA with version comment
174-
uses: appleboy/scp-action@ff85246acaad7bdce478db94a363cd2bf7c90345 #v1.0.0
24+
## Skills
17525

176-
# Bad: floating tag
177-
uses: appleboy/scp-action@v1.0.0
178-
```
26+
Load these on demand -- don't include their content in every response, just invoke them when the task calls for it.
17927

180-
Exceptions: Astral (uv) actions may use fully qualified semver tags (e.g. `astral-sh/setup-uv@7.3.0`), and immutable tags are allowed.
28+
| Skill | When to use |
29+
|---|---|
30+
| `/onboarding` | Setting up the repo for the first time or helping a new contributor |
31+
| `/backend` | Working on backend code (commands, env vars, Docker) |
32+
| `/frontend` | Working on frontend code (commands, type generation, Docker) |
33+
| `/corpus-dataset` | Adding, modifying, or reviewing `.txt` law files in `backend/scripts/documents/` (includes ASCII enforcement rules) |
34+
| `/evaluation` | Running, analyzing, or improving LangSmith evaluations |
35+
| `/pr` | Writing commit messages, opening PRs, or reviewing code |

‎.claude/settings.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@
3636
"frontend-design@claude-plugins-official": true,
3737
"github@claude-plugins-official": true,
3838
"typescript-lsp@claude-plugins-official": true,
39-
"legal@knowledge-work-plugins": true
39+
"legal@knowledge-work-plugins": true,
40+
"ultrareview@claude-plugins-official": true
4041
}
4142
}

‎.claude/skills/analyze-experiment.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
# Analyze LangSmith Experiment
22

3-
Analyze an evaluation experiment using the `langsmith_dataset` CLI.
3+
Analyze an evaluation experiment using the `langsmith_dataset` CLI. Use the `/langsmith` skills for LangSmith-specific guidance (tracing, datasets, evaluators).
44
The argument is an experiment name or UUID (e.g. `/analyze-experiment tfa-2026-04-13` or `/analyze-experiment c663e09e-...`).
55

66
## Instructions
77

88
The CLI entry point is:
99
```
10-
uv run python -m evaluate.langsmith_dataset <noun> <verb> [args]
10+
uv run langsmith-dataset <noun> <verb> [args]
1111
```
1212
Run all CLI commands from `backend/` with `dangerouslyDisableSandbox: true` (requires network access to LangSmith).
1313

‎.claude/skills/backend.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Backend
2+
3+
Reference for backend development workflow. Run all commands from `backend/`.
4+
5+
## Commands
6+
7+
```bash
8+
make fmt # Format + sort imports (ruff)
9+
make lint # Lint (ruff)
10+
make typecheck # Type-check (ty)
11+
make test # Run tests (pytest)
12+
make --keep-going check # All of the above in one shot
13+
14+
# Single test
15+
uv run pytest -s -k <test_name>
16+
17+
# LangChain-specific tests
18+
uv run pytest -m langchain
19+
20+
# Coverage
21+
make test TEST_OPTIONS="--cov tenantfirstaid --cov-report html --cov-branch"
22+
```
23+
24+
All Python commands must be run via `uv run python ...`. Use the `/uv` skill for uv-specific guidance.
25+
26+
## Environment variables
27+
28+
See `backend/.env.example`. Key ones:
29+
30+
| Variable | Description |
31+
|---|---|
32+
| `MODEL_NAME` | LLM model name (e.g. `gemini-2.5-pro`) |
33+
| `GOOGLE_APPLICATION_CREDENTIALS` | Path to GCP credentials JSON |
34+
| `VERTEX_AI_DATASTORE_LAWS` | Vertex AI Search datastore ID for the laws corpus |
35+
| `LANGSMITH_API_KEY` | LangSmith API key (required for evaluations) |
36+
| `LANGSMITH_TRACING` | Set to `true` to enable LangSmith tracing |
37+
| `LANGCHAIN_TRACING_V2` | Set to `true` for detailed tracing |
38+
| `LANGSMITH_PROJECT` | LangSmith project name for traces |
39+
40+
## LangSmith evaluations
41+
42+
```bash
43+
# Run LangChain-specific tests
44+
uv run pytest -m langchain
45+
46+
# Run with LangSmith tracing
47+
LANGSMITH_TRACING=true LANGCHAIN_TRACING_V2=true uv run pytest -m langchain
48+
49+
# Run full evaluation suite
50+
uv run run-langsmith-evaluation --num-repetitions 20
51+
```
52+
53+
See `/evaluation` for the full evaluation workflow.
54+
55+
## Docker
56+
57+
Backend Dockerfile targets (`backend/Dockerfile`):
58+
59+
| Target | Purpose |
60+
|---|---|
61+
| `runtime` | Production image (gunicorn, no dev deps) |
62+
| `ci` | Full dev environment for running checks |
63+
64+
Build a specific target:
65+
66+
```sh
67+
docker build -f backend/Dockerfile --target runtime -t tenantfirstaid-backend:runtime backend
68+
```
69+
70+
Run checks via compose:
71+
72+
```sh
73+
RUNTIME_TARGET=ci docker compose run backend uv run --dev pytest -q
74+
```

‎.claude/skills/corpus-dataset.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Corpus Dataset
2+
3+
Reference for adding or modifying law documents in `backend/scripts/documents/`.
4+
5+
## ASCII requirement
6+
7+
All `.txt` files must be **pure ASCII**. Vertex AI RAG ingestion has produced mojibake when UTF-8 is present. Before committing any `.txt` file, verify:
8+
9+
```bash
10+
LC_ALL=C grep -n '[^[:print:][:space:]]' path/to/file.txt # must produce no output
11+
```
12+
13+
`make generate-metadata` auto-converts all known offenders in place and warns with a suggested replacement for any it cannot handle.
14+
15+
### Common replacements
16+
17+
| Character | Unicode | Replace with |
18+
|---|---|---|
19+
| `'` | U+2019 right single quote | `'` |
20+
| `"` `"` | U+201C/U+201D double quotes | `"` |
21+
| `§` | U+00A7 section sign | `Section ` |
22+
| `§§` | U+00A7 U+00A7 | `Sections ` |
23+
| `—` | U+2014 em dash | `--` |
24+
| `–` | U+2013 en dash | `-` |
25+
| `•` | U+2022 bullet | `-` |
26+
27+
See `ASCII_REPLACEMENTS` in `backend/scripts/generate_metadata_jsonl.py` for the full list.
28+
29+
## Publication cadence
30+
31+
Oregon publishes statutes and annotations on offset biennial cycles:
32+
33+
- **Statutes** (`ORS*.txt`, `OAR*.txt`) — published in odd years after each long legislative session. [Current edition](https://www.oregonlegislature.gov/bills_laws/pages/ors.aspx).
34+
- **Annotations** (`ORS*_annotations.txt`) — published in the fall of each even-numbered year as a Cumulative Supplement. [Current edition](https://www.oregonlegislature.gov/bills_laws/Pages/Annotations.aspx).
35+
36+
This is why statutes and annotations live in different year subfolders.
37+
38+
## URL patterns
39+
40+
Chapter pages are at predictable URLs (`NNN` = three-digit zero-padded chapter number, lowercase letter suffix for sub-chapters):
41+
42+
- Statutes: `https://www.oregonlegislature.gov/bills_laws/ors/orsNNN.html`
43+
- Annotations: `https://www.oregonlegislature.gov/bills_laws/ors/anoNNN.html`
44+
45+
Examples: chapter 90 → `ors090.html` / `ano090.html`; chapter 659A → `ors659a.html` / `ano659a.html`.

0 commit comments

Comments
 (0)