Akvo Chat Service

Standalone RAG chat service with hybrid search and LLM answer generation. Designed to be reusable across multiple Akvo projects.

Architecture

Markdown / RST / PDF / URLs
       |  (python ingest CLI)
       v
 Weaviate (vector DB)
 - text-embedding-3-small (vectorisation)
 - Hybrid search (vector + BM25)
       |
       v
 FastAPI Service (:8100)
 - Hybrid retrieval (top-K chunks)
 - Answer generation (gpt-4o-mini)
 - Thread persistence (PostgreSQL)
 - API-key auth per project

Quick Start

# 1. Environment
cp .env.example .env
# Edit .env — set OPENAI_API_KEY at minimum

# 2. Start the stack
docker compose up -d --build

# 3. Verify
curl http://localhost:8100/v1/health

# 4. Ingest documentation
docker compose exec -T chat-api python -m chat_service.ingest markdown ./docs/ \
  --collection my-project --skip-enrichment

# 5. Send a message
curl -X POST http://localhost:8100/v1/chat/message \
  -H "Authorization: Bearer my-dev-key-123" \
  -H "Content-Type: application/json" \
  -d '{"message": "How do I get started?", "collection": "my-project"}'

Services

Service	Port	Purpose
chat-api	8100	FastAPI application
db	5433	PostgreSQL (thread persistence)
weaviate	8080	Vector database
reranker	—	Cross-encoder sidecar (optional, CPU)

API Endpoints

Chat

Method	Path	Auth	Purpose
`POST`	`/v1/chat/message`	Yes	Send a message, get an answer
`GET`	`/v1/chat/threads`	Yes	List conversation threads
`GET`	`/v1/chat/threads/{id}`	Yes	Get thread with message history
`DELETE`	`/v1/chat/threads/{id}`	Yes	Delete a thread

Admin

Method	Path	Auth	Purpose
`GET`	`/v1/health`	No	Health check
`GET`	`/v1/collections`	No	List Weaviate collections

Example: Send a message

curl -X POST http://localhost:8100/v1/chat/message \
  -H "Authorization: Bearer my-dev-key-123" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What can I do on this page?",
    "page_context": "Control Center — Form Builder",
    "collection": "mis"
  }'

Response:

{
  "response": "On the Form Builder page you can create, edit, publish and manage forms...",
  "thread_id": "thr_a1b2c3d4e5f6...",
  "sources": [
    {"title": "Form Builder", "url": "formBuilder.rst", "score": 5.23}
  ]
}

Request Fields

Field	Type	Required	Description
`message`	string	Yes	The user's question
`collection`	string	No	Weaviate collection (defaults to API key's collection)
`page_context`	string	No	Current page label — improves retrieval
`thread_id`	string	No	Continue an existing conversation thread

Knowledge Ingestion

The CLI supports Markdown and reStructuredText files.

# Ingest markdown files
docker compose exec -T chat-api python -m chat_service.ingest markdown ./docs/ \
  --collection mis --skip-enrichment

# Ingest RST files (e.g. Sphinx documentation)
docker compose exec -T chat-api python -m chat_service.ingest rst ./docs/source/ \
  --collection mis --skip-enrichment

# With contextual enrichment (slower, better retrieval accuracy)
docker compose exec -T chat-api python -m chat_service.ingest markdown ./docs/ \
  --collection mis

# Preview without indexing
docker compose exec -T chat-api python -m chat_service.ingest markdown ./docs/ \
  --collection mis --dry-run

# List collections and chunk counts
docker compose exec -T chat-api python -m chat_service.ingest list

# Drop a collection (re-index from scratch)
docker compose exec -T chat-api python -m chat_service.ingest drop --collection mis

Ingestion Options

Flag	Purpose
`--collection`	Target Weaviate collection name
`--source-type`	Label for source type (default: `docs`)
`--tags`	Comma-separated tags for filtering
`--skip-enrichment`	Skip LLM contextual enrichment (faster)
`--dry-run`	Preview chunk counts without indexing

Copying Docs into the Container

When docs live outside the chat service repo:

# Copy docs from host into the running container
docker compose cp /path/to/your/docs chat-api:/tmp/docs

# Then ingest
docker compose exec -T chat-api python -m chat_service.ingest rst /tmp/docs \
  --collection my-project --skip-enrichment

Configuration

Variable	Default	Purpose
`OPENAI_API_KEY`	required	OpenAI API key for embeddings + LLM
`OPENAI_ANSWER_MODEL`	`gpt-4o-mini`	Answer generation model
`OPENAI_PLANNER_MODEL`	`gpt-4o-mini`	Query planning + enrichment model
`DATABASE_URL`	required	PostgreSQL connection string
`WEAVIATE_URL`	`http://weaviate:8080`	Weaviate HTTP endpoint
`WEAVIATE_GRPC_URL`	`weaviate:50051`	Weaviate gRPC endpoint
`RETRIEVAL_MAX_VARIANTS`	`1`	Search variants (1 = no planner)
`RETRIEVAL_LIMIT_PER_QUERY`	`6`	Results per search query
`RETRIEVAL_TOP_K`	`8`	Chunks sent to LLM for generation
`RETRIEVAL_ALPHA`	`0.75`	Hybrid search balance (1=vector, 0=BM25)
`CHAT_API_KEYS`	—	Production keys: `key1:col1,key2:col2`
`CHAT_DEV_API_KEY`	—	Development API key
`CHAT_DEV_COLLECTION`	`default`	Collection for dev key
`CORS_ORIGINS`	`*`	Comma-separated allowed origins
`THREAD_RETENTION_DAYS`	`90`	Auto-cleanup threads older than N days

Authentication

API keys are registered via environment variables:

# Single dev key
CHAT_DEV_API_KEY=my-dev-key-123
CHAT_DEV_COLLECTION=mis

# Multiple production keys (key:collection pairs)
CHAT_API_KEYS=proj1-key-abc:project1,proj2-key-xyz:project2

Each key is bound to a collection. Include the key in requests:

Authorization: Bearer my-dev-key-123

Integrating with Any Project

This service is designed to be reused across multiple projects. Each project gets its own API key and isolated Weaviate collection.

Step 1: Register an API Key

Add a key-collection pair to the chat service .env:

# Single project
CHAT_API_KEYS=myproject-key-abc:myproject

# Multiple projects sharing one service
CHAT_API_KEYS=mis-key-123:mis,lumen-key-456:lumen,rush-key-789:rush

Restart the service to pick up new keys:

docker compose up -d chat-api

Step 2: Ingest Project Documentation

# Copy docs into the container
docker compose cp /path/to/project/docs chat-api:/tmp/docs

# Ingest markdown or RST
docker compose exec -T chat-api python -m chat_service.ingest markdown /tmp/docs \
  --collection myproject --skip-enrichment

Step 3: Call the API

From any frontend or backend — just an HTTP POST:

curl -X POST https://chat.akvo.org/v1/chat/message \
  -H "Authorization: Bearer myproject-key-abc" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "How do I export data?",
    "page_context": "Dashboard",
    "collection": "myproject"
  }'

Data Isolation

Each project's data is fully isolated:

┌──────────────────────────────────────┐
│        Chat Service (:8100)          │
│                                      │
│  API Key: mis-key    → Collection: mis
│  API Key: lumen-key  → Collection: lumen
│  API Key: rush-key   → Collection: rush
│                                      │
│  Threads scoped per key (PostgreSQL) │
└──────────────────────────────────────┘

Collections are independent — ingesting docs for one project doesn't affect another
Threads are scoped per API key — projects can't see each other's conversations
API keys can be rotated independently

Frontend Integration Patterns

Pattern A: Dev Proxy (React / Vue / Next.js)

Route requests through the frontend dev server proxy to avoid CORS:

// React (setupProxy.js)
app.use("/chat-api", createProxyMiddleware({
  target: "http://host.docker.internal:8100",  // Docker-to-host
  changeOrigin: true,
  pathRewrite: { "^/chat-api": "" },
}));

// Vite (vite.config.js)
export default defineConfig({
  server: {
    proxy: {
      "/chat-api": {
        target: "http://host.docker.internal:8100",
        changeOrigin: true,
        rewrite: (path) => path.replace(/^\/chat-api/, ""),
      },
    },
  },
});

Then call from the frontend:

const res = await axios.post("/chat-api/v1/chat/message", {
  message: userInput,
  collection: "myproject",
  page_context: currentPage,
}, {
  headers: { Authorization: "Bearer myproject-key-abc" },
});

Pattern B: Direct API Call

No proxy needed — call the chat service directly (requires CORS to be configured):

const CHAT_URL = "https://chat.akvo.org/v1/chat";
const API_KEY = "myproject-key-abc";

const res = await fetch(`${CHAT_URL}/message`, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${API_KEY}`,
  },
  body: JSON.stringify({
    message: userInput,
    collection: "myproject",
    page_context: "Settings Page",
  }),
});
const data = await res.json();

Pattern C: Backend Proxy (hides API key)

Route through your own backend so the chat API key never reaches the browser:

# Django / Flask / FastAPI
import httpx

CHAT_SERVICE_URL = "http://chat-service:8100"
CHAT_API_KEY = "myproject-key-abc"

@app.post("/api/chat")
def chat_proxy(request):
    res = httpx.post(
        f"{CHAT_SERVICE_URL}/v1/chat/message",
        json={
            "message": request.data["message"],
            "collection": "myproject",
            "page_context": request.data.get("page_context"),
            "thread_id": request.data.get("thread_id"),
        },
        headers={"Authorization": f"Bearer {CHAT_API_KEY}"},
    )
    return res.json()

Akvo MIS Integration (Reference)

The MIS frontend uses Pattern A with a ChatbotWidget component:

// frontend/src/setupProxy.js
app.use("/chat-api", createProxyMiddleware({
  target: "http://host.docker.internal:8100",
  changeOrigin: true,
  pathRewrite: { "^/chat-api": "" },
}));

The widget sends the user's current page as page_context (e.g. "Control Center — Form Builder") for context-aware retrieval, and persists thread_id in sessionStorage for conversation continuity.

Running Tests

docker compose exec -T chat-api pytest tests/ -v

Project Structure

chat_service/
  main.py              # FastAPI app, CORS, startup
  config.py            # Pydantic settings from env
  db.py                # SQLAlchemy engine + session
  weaviate_client.py   # Weaviate connection helper
  api/
    routes_chat.py     # POST /v1/chat/message, threads CRUD
    routes_admin.py    # GET /v1/health, /v1/collections
    auth.py            # API-key registration + verification
    schemas.py         # Pydantic request/response models
  retrieval/
    planner.py         # Query variant generation (optional)
    searcher.py        # Weaviate hybrid search
    reranker.py        # Cross-encoder reranking (optional)
    pipeline.py        # Retrieval orchestration
  generation/
    generator.py       # LLM answer generation
    prompts.py         # System prompt template
  threads/
    models.py          # SQLAlchemy models (ChatThread, ChatThreadMessage)
    service.py         # Thread CRUD operations
    naming.py          # Thread title generation
  ingest/
    __main__.py        # CLI entry point (markdown, rst, list, drop)
    chunking.py        # Word-based text chunking (300 words, 50 overlap)
    contextual.py      # LLM contextual enrichment (optional)
    sources/
      markdown.py      # Markdown file loader
      rst.py           # reStructuredText file loader
      pdf.py           # PDF loader (placeholder)
      url.py           # URL crawler (placeholder)
tests/
  test_chunking.py     # Chunking unit tests

Cost Estimate

With gpt-4o-mini and self-hosted Weaviate:

Volume	Estimated cost
1,000 msgs/month	~$0.54
10,000 msgs/month	~$5.40

No vector store fees — Weaviate and PostgreSQL are self-hosted. The only cost is OpenAI API usage for embeddings (at ingestion) and answer generation (per message).

Name		Name	Last commit message	Last commit date
chat_service		chat_service
tests		tests
.env.example		.env.example
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
Dockerfile		Dockerfile
README.md		README.md
docker-compose.yml		docker-compose.yml
requirements.txt		requirements.txt

Navigation Menu

Uh oh!

Repository files navigation

Akvo Chat Service

Architecture

Quick Start

Services

API Endpoints

Chat

Admin

Example: Send a message

Request Fields

Knowledge Ingestion

Ingestion Options

Copying Docs into the Container

Configuration

Authentication

Integrating with Any Project

Step 1: Register an API Key

Step 2: Ingest Project Documentation

Step 3: Call the API

Data Isolation

Frontend Integration Patterns

Pattern A: Dev Proxy (React / Vue / Next.js)

Pattern B: Direct API Call

Pattern C: Backend Proxy (hides API key)

Akvo MIS Integration (Reference)

Running Tests

Project Structure

Cost Estimate

About

Releases

Packages

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

Akvo Chat Service

Architecture

Quick Start

Services

API Endpoints

Chat

Admin

Example: Send a message

Request Fields

Knowledge Ingestion

Ingestion Options

Copying Docs into the Container

Configuration

Authentication

Integrating with Any Project

Step 1: Register an API Key

Step 2: Ingest Project Documentation

Step 3: Call the API

Data Isolation

Frontend Integration Patterns

Pattern A: Dev Proxy (React / Vue / Next.js)

Pattern B: Direct API Call

Pattern C: Backend Proxy (hides API key)

Akvo MIS Integration (Reference)

Running Tests

Project Structure

Cost Estimate

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages