|
| 1 | +# Per-User System Prompt |
| 2 | + |
| 3 | +One agent, one deployment — but every user gets their own system prompt, keyed |
| 4 | +by `user_id` in a compacted Kafka topic. A user with no record falls back to the |
| 5 | +agent's default prompt, which here is deliberately useless: it answers |
| 6 | +`I don't know.` to everything. |
| 7 | + |
| 8 | +## What it does |
| 9 | + |
| 10 | +Two users are seeded into `{AGENT_NAME}-user-settings`, a third is left out: |
| 11 | + |
| 12 | +| user_id | system prompt | answer to *"my orders table is slow, what should I do?"* | |
| 13 | +|---------|---------------|-----------------------------------------------------| |
| 14 | +| `user_alice` | SQL tutor persona (seeded) | `[SQL-TUTOR] Add an index on …` | |
| 15 | +| `user_bob` | haiku poet persona (seeded) | `[HAIKU]` + a three-line haiku | |
| 16 | +| `user_carol` | *(no record)* | `I don't know.` | |
| 17 | + |
| 18 | +Same agent, same question, same session pipeline — the only difference is which |
| 19 | +system prompt the think-consumer resolves for that user. |
| 20 | + |
| 21 | +## How it works |
| 22 | + |
| 23 | +``` |
| 24 | +POST /api/chat {user_id: "user_alice", ...} |
| 25 | + │ |
| 26 | + ▼ |
| 27 | + message-input ──► processing (Kafka Streams) |
| 28 | + │ |
| 29 | + │ re-key by user_id |
| 30 | + │ left-join {AGENT_NAME}-user-settings (KTable, compacted) |
| 31 | + │ re-key back to session_id |
| 32 | + ▼ |
| 33 | + enriched-message-input { …, "system_prompt": "You are Ada …" } |
| 34 | + │ |
| 35 | + ▼ |
| 36 | + think-consumer |
| 37 | + system_prompt present ? use it : use SYSTEM_PROMPT_FILE |
| 38 | +``` |
| 39 | + |
| 40 | +**The topic.** `{AGENT_NAME}-user-settings` is compacted and keyed by `user_id`: |
| 41 | + |
| 42 | +``` |
| 43 | +key: user_alice |
| 44 | +value: {"system_prompt":"You are Ada, a senior database engineer …","updated_at":"2026-08-23T00:00:00Z"} |
| 45 | +``` |
| 46 | + |
| 47 | +Compaction means the topic *is* the current settings table: the latest record |
| 48 | +per user is retained forever, and a tombstone (null value) deletes a user's |
| 49 | +override so they revert to the default prompt. |
| 50 | + |
| 51 | +**The join.** With `ENABLE_USER_SETTING=true`, the processing app creates the |
| 52 | +topic if missing, materializes it as a `KTable`, and left-joins it into |
| 53 | +`FullSessionContext` during enrichment. Left-join is the point: a user with no |
| 54 | +record still flows through, just with `system_prompt: null`. |
| 55 | + |
| 56 | +**The resolution.** In the think-consumer, a non-blank `system_prompt` from the |
| 57 | +context *replaces* the base prompt (`SYSTEM_PROMPT_FILE`, here |
| 58 | +[`system-prompt.txt`](./system-prompt.txt)) entirely — it is not appended to it. |
| 59 | +Null or blank falls back to the file. |
| 60 | + |
| 61 | +**Timing.** The KTable is read at message-flow time, so a settings update takes |
| 62 | +effect on the user's **next** turn, not the one already in flight. |
| 63 | + |
| 64 | +## Configuration |
| 65 | + |
| 66 | +| Parameter | Service | Value | Why | |
| 67 | +|-----------|---------|-------|-----| |
| 68 | +| `ENABLE_USER_SETTING` | `processing` | `true` | Creates the topic, materializes the KTable, joins it into the session context | |
| 69 | +| `MEMOIR_ENABLED` | `processing` | `false` | Off, so the only per-user difference is the system prompt | |
| 70 | +| `SYSTEM_PROMPT_FILE` | `think-consumer` | `/app/system-prompt.txt` | The default for users with no settings record | |
| 71 | + |
| 72 | +`ENABLE_USER_SETTING` defaults to `false`; with it off, the topology never |
| 73 | +references the topic at all and `system_prompt` stays null for everyone. |
| 74 | + |
| 75 | +## Services |
| 76 | + |
| 77 | +`chat-api`, `processing` and `think-consumer` are **built from source** — the |
| 78 | +per-user system prompt is not in the published images yet, and all three must |
| 79 | +share one build. Only the frontend uses a published image. |
| 80 | + |
| 81 | +| Service | Description | |
| 82 | +|---------|-------------| |
| 83 | +| `api` | Chat API (REST + WebSocket) | |
| 84 | +| `processing` | Kafka Streams pipeline with `ENABLE_USER_SETTING=true` | |
| 85 | +| `think-consumer` | Claude API caller; resolves the per-user prompt | |
| 86 | +| `user-settings-seed` | One-shot job that writes the two seeded users, then exits | |
| 87 | +| `frontend` | Web UI | |
| 88 | + |
| 89 | +## Run |
| 90 | + |
| 91 | +```bash |
| 92 | +cp .env.example .env |
| 93 | +# Edit .env and add your CLAUDE_API_KEY |
| 94 | + |
| 95 | +docker compose up --build |
| 96 | +``` |
| 97 | + |
| 98 | +Ask the same question as each user: |
| 99 | + |
| 100 | +```bash |
| 101 | +curl -s localhost:8000/api/chat -H 'Content-Type: application/json' -d '{ |
| 102 | + "session_id": "s-alice-1", "user_id": "user_alice", |
| 103 | + "content": "My orders table has 50 million rows and one query over it is slow. What should I do?"}' |
| 104 | + |
| 105 | +curl -s localhost:8000/api/chat -H 'Content-Type: application/json' -d '{ |
| 106 | + "session_id": "s-bob-1", "user_id": "user_bob", |
| 107 | + "content": "My orders table has 50 million rows and one query over it is slow. What should I do?"}' |
| 108 | + |
| 109 | +curl -s localhost:8000/api/chat -H 'Content-Type: application/json' -d '{ |
| 110 | + "session_id": "s-carol-1", "user_id": "user_carol", |
| 111 | + "content": "My orders table has 50 million rows and one query over it is slow. What should I do?"}' |
| 112 | +``` |
| 113 | + |
| 114 | +Read the answers off the output topic: |
| 115 | + |
| 116 | +```bash |
| 117 | +docker compose exec kafka /opt/kafka/bin/kafka-console-consumer.sh \ |
| 118 | + --bootstrap-server localhost:9092 \ |
| 119 | + --topic per-user-prompt-message-output --from-beginning --timeout-ms 5000 |
| 120 | +``` |
| 121 | + |
| 122 | +Alice gets a terse SQL answer tagged `[SQL-TUTOR]`, Bob gets a haiku tagged |
| 123 | +`[HAIKU]`, and Carol — who has no settings record — gets `I don't know.` |
| 124 | + |
| 125 | +The compose file defaults to `claude-opus-5`; override with `CLAUDE_MODEL` in |
| 126 | +`.env` (and adjust `INPUT_TOKEN_PRICE` / `OUTPUT_TOKEN_PRICE`, which are per 1M |
| 127 | +tokens, to match). |
| 128 | + |
| 129 | +### From the browser |
| 130 | + |
| 131 | +The web UI at [http://localhost](http://localhost) does not send a `user_id`, so |
| 132 | +chat-api defaults it to `user_42`, which is unseeded — the UI will answer |
| 133 | +`I don't know.` until you give that user a prompt: |
| 134 | + |
| 135 | +```bash |
| 136 | +docker compose exec -T kafka /opt/kafka/bin/kafka-console-producer.sh \ |
| 137 | + --bootstrap-server localhost:9092 \ |
| 138 | + --topic per-user-prompt-user-settings --property parse.key=true <<'REC' |
| 139 | +user_42 {"system_prompt":"You are Rex, a pirate captain. Answer everything in pirate speak, in two sentences or less.","updated_at":"2026-08-23T00:00:00Z"} |
| 140 | +REC |
| 141 | +``` |
| 142 | + |
| 143 | +(The key and the JSON value are separated by a literal **tab**.) Send another |
| 144 | +message from the UI and the pirate answers. |
| 145 | + |
| 146 | +### Reverting a user |
| 147 | + |
| 148 | +A tombstone — a record with a null value — drops the override, and that user |
| 149 | +goes back to the default prompt: |
| 150 | + |
| 151 | +```bash |
| 152 | +docker compose exec -T kafka /opt/kafka/bin/kafka-console-producer.sh \ |
| 153 | + --bootstrap-server localhost:9092 \ |
| 154 | + --topic per-user-prompt-user-settings \ |
| 155 | + --property parse.key=true --property null.marker=NULL <<'REC' |
| 156 | +user_42 NULL |
| 157 | +REC |
| 158 | +``` |
| 159 | + |
| 160 | +### Inspecting the settings table |
| 161 | + |
| 162 | +```bash |
| 163 | +docker compose exec kafka /opt/kafka/bin/kafka-console-consumer.sh \ |
| 164 | + --bootstrap-server localhost:9092 \ |
| 165 | + --topic per-user-prompt-user-settings --from-beginning \ |
| 166 | + --property print.key=true --timeout-ms 5000 |
| 167 | +``` |
| 168 | + |
| 169 | +## Seeding your own users |
| 170 | + |
| 171 | +[`seed-data/seed-user-settings.sh`](./seed-data/seed-user-settings.sh) is the |
| 172 | +one-shot seed job: it waits for the compacted topic to exist, then produces one |
| 173 | +keyed record per user. Add a user by adding a prompt variable and a `printf` |
| 174 | +line — prompts are JSON string bodies, so use `\n` for line breaks and avoid raw |
| 175 | +double quotes. |
| 176 | + |
| 177 | +In a real deployment this topic is written by whatever owns user preferences — |
| 178 | +an admin UI, a settings service, a CDC stream off your users table — not by a |
| 179 | +shell script. |
| 180 | + |
| 181 | +## Automated test |
| 182 | + |
| 183 | +```bash |
| 184 | +CLAUDE_API_KEY=sk-ant-... ./integration-test.sh |
| 185 | +``` |
| 186 | + |
| 187 | +Brings the stack up, waits for the seed, asks all three users the same question, |
| 188 | +and asserts each answered under its own prompt (and that Carol fell back to |
| 189 | +`I don't know.`). Tears the stack down on exit. |
0 commit comments