feat: editable LLM prompts and per-family preferred model by MkDev11 · Pull Request #951 · we-promise/sure

MkDev11 · 2026-02-09T15:35:12Z

What this does

The AI Prompts settings page was always read-only. This PR makes it actually useful: you can now edit the main system prompt, the intro (welcome) prompt, and choose a default AI model for your household. Each family can tune how the assistant behaves and which model new chats use.

Changes

Family settings — New columns: custom_system_prompt, custom_intro_prompt, preferred_ai_model. All optional; leave blank to keep the built-in defaults.
Assistant behavior — When you chat, the assistant uses your family’s custom prompts when set, otherwise the existing defaults (including the intro flow from PR First cut of a simplified "intro" UI layout #265).
Default model — New chats and messages use your family’s preferred model when set, then fall back to the existing global default (ENV / Setting / OpenAI default).
Settings UI — The AI Prompts page now has a form at the top to edit those three fields, with basic validation (length limits) and error handling.
Tests — Controller tests for the update flow; Chat tests updated so default-model behavior is stable. Added sidekiq-throttled to the Gemfile so the test suite can boot (required by IndexaCapitalActivitiesFetchJob).

Why

So each household can adjust the assistant’s personality and target model without touching code or config files. Matches the intent behind #938 and the discussion in #906.

How to verify

As an admin, go to Settings → AI Prompts.
Set a preferred model (e.g. gpt-4-turbo), optionally edit the main or intro prompt, and save.
Start a new chat without picking a model — it should use your preferred model.
The collapsible “Prompt Instructions” sections below should show the effective prompt (your custom one or the default).

Run the tests:

bin/rails test test/controllers/settings/ai_prompts_controller_test.rb test/models/chat_test.rb test/models/family_test.rb

Closes #938.

Summary by CodeRabbit

New Features
- Families can set custom AI system and intro prompts, preferred AI model, and custom OpenAI endpoint.
- Settings UI now lets admins view and update per-family AI prompt and model overrides.
- AI model selection now honors per-family preferences when present.
Chores
- Added background job throttling dependency.

- Add .cursor/rules/api-endpoint-consistency.mdc: checklist that applies when editing app/controllers/api/v1, spec/requests/api/v1, or test/controllers/api/v1. Enforces (1) Minitest-only behavioral coverage for new endpoints, (2) rswag docs-only (no expect/assert), (3) same API key auth pattern in all rswag specs. - Reference the rule in AGENTS.md under API Development Guidelines.

- Minitest: test/api_endpoint_consistency_rule_test.rb checks rule file exists, globs, and all three sections (Minitest, rswag docs-only, API key auth) plus AGENTS.md reference. - Standalone: test/support/verify_api_endpoint_consistency.rb runs same checks without loading Rails (use when app fails to boot). - Rule: add mdc: links for Cursor, note valuations_spec OAuth outlier.

…mise#938) - Add family columns: custom_system_prompt, custom_intro_prompt, preferred_ai_model - Assistant::Configurable uses family overrides when present, else built-in defaults - Chat.default_model(family) supports per-family preferred model; controllers pass family - Settings AI Prompts page: form to edit custom prompts and preferred model, validation - Route and Settings::AiPromptsController#update with strong params and error handling - Family validations: prompt length cap 32k chars, preferred_ai_model 128 chars

…er tests - Gemfile: add sidekiq-throttled (required by IndexaCapitalActivitiesFetchJob) - ChatTest: stub ENV/Setting and clear family preferred_ai_model for default-model tests - AiPromptsControllerTest: drop assigns/assert_template; assert response content and non-persistence

coderabbitai · 2026-02-09T15:35:33Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds per-family AI prompt and model overrides, custom OpenAI endpoint support, family-aware model/provider resolution, admin-editable settings, database columns for family overrides, controller and helper updates to use family context, provider registry changes, and related tests and UI/locales.

Changes

Cohort / File(s)	Summary
Database migrations & schema `db/migrate/20260209120000_add_ai_prompt_overrides_to_families.rb`, `db/migrate/20260209180000_add_openai_uri_base_to_families.rb`, `db/schema.rb`	Adds family-level columns: `custom_system_prompt`, `custom_intro_prompt`, `preferred_ai_model`, `openai_uri_base`. Schema reflects added columns and other unrelated reorders/removals.
Family model & validations `app/models/family.rb`	Adds length constants, validations for new prompt/model/URI fields, `custom_openai_endpoint?` predicate, and cross-field validation enforcing `preferred_ai_model` when `openai_uri_base` is set.
Chat default model & helpers `app/models/chat.rb`, `app/helpers/application_helper.rb`	`Chat.default_model` now accepts optional `family` and prioritizes `family.preferred_ai_model`; helper `default_ai_model` now passes `Current.user&.family`.
Controllers — model resolution changes `app/controllers/chats_controller.rb`, `app/controllers/messages_controller.rb`, `app/controllers/api/v1/chats_controller.rb`, `app/controllers/api/v1/messages_controller.rb`	Controller call sites updated to use family-aware default model (`Chat.default_model(family)`) when `ai_model` not provided.
Settings UI, controller, routes, locales `app/controllers/settings/ai_prompts_controller.rb`, `app/views/settings/ai_prompts/show.html.erb`, `config/routes.rb`, `config/locales/views/settings/en.yml`	Adds admin-only `update` action, `set_family` and strong params, expands routes to allow `show` and `update`, adds form UI and i18n keys for editing family prompts, model, and endpoint.
Provider registry & assistant/provider plumbing `app/models/provider/registry.rb`, `app/models/assistant/provided.rb`, `app/models/assistant.rb`, `app/models/assistant/configurable.rb`	Adds `Provider::Registry.openai_for_family(family)` to build family-scoped OpenAI provider; provider lookup accepts `family:` and prefers family-scoped provider; Assistant/Configurable use family prompts and pass family to provider lookup.
Dependencies `Gemfile`	Adds `sidekiq-throttled` gem.
Tests `test/controllers/settings/ai_prompts_controller_test.rb`, `test/models/chat_test.rb`, `test/models/assistant_test.rb`	Adds controller tests for AI prompts (authorization, persistence, validation, reset); updates tests to account for family-aware provider/model lookup and ENV/Setting fallbacks.
Other model changes `app/models/assistant/configurable.rb`, `app/models/assistant/provided.rb`, `app/models/assistant.rb`	Refactors prompt selection to prefer family custom prompts and threads family through model/provider selection.

Sequence Diagram

sequenceDiagram
    participant Browser as User (Browser)
    participant Controller as ChatsController / MessagesController
    participant Chat as Chat Model
    participant Family as Family Record
    participant Registry as Provider::Registry
    participant Provider as Provider::Openai
    participant Assistant as Assistant

    Browser->>Controller: POST /chats or /messages (no ai_model)
    Controller->>Chat: Chat.default_model(Current.user.family)
    Chat->>Family: read preferred_ai_model/custom prompts
    alt family has preferred_ai_model
        Family-->>Chat: family preferred_ai_model
    else
        Chat-->>Chat: fallback to ENV/Setting/default
    end
    Chat-->>Controller: resolved ai_model

    Controller->>Assistant: create/message with ai_model and family
    Assistant->>Registry: get_model_provider(ai_model, family: family)
    Registry->>Family: check openai_uri_base & preferred_ai_model
    alt family has custom endpoint & token
        Registry->>Provider: instantiate Provider::Openai(custom uri, model)
    else
        Registry->>Provider: return default provider
    end
    Provider-->>Assistant: provider instance
    Assistant->>Assistant: apply family custom prompts if present
    Assistant-->>Browser: AI response

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~35 minutes

Possibly related PRs

Fix "Messages is invalid" error for Ollama/custom LLM providers and add comprehensive AI documentation #225 — overlaps on making default-model resolution family-aware and updating controller call sites.
Implement support for generic OpenAI api #213 — related provider/OpenAI configuration changes and uri_base handling used by the registry.
Lunchflow settings family #363 — related family-scoped provider construction and settings plumbing.

Suggested labels

enhancement

Suggested reviewers

sokie

Poem

🐰 I nibble prompts with careful care,
Families choose their model, fair and square.
Endpoints tuned and settings set just right,
Custom prompts bloom in morning light.
Hop, hop — AI chats now feel delight! ✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 6.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Out of Scope Changes check	❓ Inconclusive	The dependency addition (sidekiq-throttled) and schema cleanup (trades/holdings columns) extend beyond `#938` scope but are justified as supporting infrastructure.	Clarify whether trades/holdings column removal and sidekiq dependency are intentional scope or should be separated into focused PRs.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title accurately describes the main feature: editable LLM prompts and per-family preferred model selection, matching the core changes throughout the codebase.
Linked Issues check	✅ Passed	All requirements from issue `#938` are met: families can customize system and intro prompts, select a default AI model, and the UI provides editing with persistence.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d33c85c384

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

app/controllers/settings/ai_prompts_controller.rb

… tests - Add family.openai_uri_base; when set, chat uses this endpoint for that household - Provider::Registry.openai_for_family(family) builds provider from family endpoint and model (public) - Assistant uses family provider when present and supports model; validation requires model when endpoint set - AI Prompts UI: endpoint field, locale keys; controller permits openai_uri_base - Fix: openai_for_family moved to public class method (was private, would raise) - Assistant tests: expect get_model_provider(ai_model, family:)

coderabbitai · 2026-02-09T16:07:39Z

Caution