Conversational assistant for the clients of Fideicomiso X. It lets clients check their account status, report payments by sending receipts over WhatsApp, and escalate queries to a human advisor. Receipts are validated automatically with AI and archived in Google Drive. The bot also sends proactive reminders for upcoming and overdue installments through a daily cron.
Note on language: the bot's user-facing responses are in Spanish, since its clients are Spanish-speaking. All of that response copy lives in
app/core/templates.pyand is intentionally left in Spanish — it is product copy, not code. This README (developer-facing documentation) is in English.
Portfolio note: this is a sanitized public fork of a bot originally built for a client. The client's real identity, credentials, and infrastructure addresses have been replaced with neutral placeholders (
Fideicomiso X,example.com, RFC-reserved IPs, etc.). The architecture and code are intact.
- Language: Python 3.12
- Web framework: FastAPI (async) on Uvicorn
- Database: PostgreSQL managed by Supabase (accessed via
asyncpg) - Channel: WhatsApp Business Cloud API (Meta Graph API v25.0)
- AI: OpenAI (GPT-4o-mini) — intent classification and receipt validation with vision
- Storage: Google Drive API v3 (Service Account + Shared Drive)
- PDF extraction: PyMuPDF
- Logging:
python-json-logger(structured logs) - Cron: host cron (
/etc/cron.d/) on the production VM - Deploy: AWS Lightsail (Ubuntu 22.04) + Docker Compose + Caddy 2 (automatic Let's Encrypt TLS)
Exact dependencies in pyproject.toml.
WhatsApp client Cron (host cron on Lightsail, Mon–Fri 12:00 UTC)
│ │
▼ ▼
Meta Graph API ──► POST /webhook/whatsapp POST /internal/notifications/run
│ │
▼ ▼
HMAC verification X-Internal-Secret header
│ │
▼ ▼
┌──────────┴──────────┐ Enqueue upcoming/overdue installments
│ Text? Doc? Btn? │ into outbox `notificaciones`
▼ ▼ │
Fast-path keywords Vision validation (OpenAI) ▼
+ LLM classifier │ Drain outbox →
(intent + confidence) ▼ send_whatsapp_template
│ Upload to Google Drive │
▼ │ ▼
Conversational FSM ▼ Reconcile via
(app/core/conversation.py) Insert into Meta status webhook
│ pagos_reportados
▼
Response via Meta API
Flow summary:
- Meta sends every message to
POST /webhook/whatsappwith an HMAC signature. - The bot validates the signature, identifies the client by phone number, and enters a transactional lock on
user_statesto avoid race conditions. - Based on the current state and the detected intent (digit shortcut, button reply, opt-out keyword, or LLM), it produces a response and/or advances the FSM.
- Documents are processed in the background: AI validation, upload to Drive, persistence in the DB.
- In parallel, a daily cron invokes
/internal/notifications/run, which enqueues reminders and dispatches the pending ones via WhatsApp templates.
app/
├── main.py # FastAPI app + lifespan + logging
├── config.py # Settings (Pydantic) from .env
├── db.py # asyncpg pool + per-phone_number lock
├── routers/
│ ├── whatsapp.py # WhatsApp webhook (GET verify + POST)
│ └── internal.py # Internal cron endpoints (X-Internal-Secret)
├── core/
│ ├── conversation.py # FSM + intent resolution + opt-out
│ ├── notifications_policy.py # Business days, holidays, offset parsing
│ └── templates.py # Spanish response copy (all replies)
├── models/
│ ├── message.py # WhatsApp and internal message schemas
│ └── notifications.py # Pydantic schemas for the /test endpoint
└── services/
├── intent_classifier.py # LLM: free text → Intent enum
├── document_ai.py # Vision: validates receipt + extracts data
├── drive.py # Upload to Google Drive
├── pagos_reportados.py # Queries: clientes, cuotas, pagos, opt-out
├── notifications.py # Outbox: enqueue, drain, reconcile
├── whatsapp_sender.py # Message sending / interactive menu / templates
├── whatsapp_media.py # Downloading files from WhatsApp
├── escalations.py # Human-advisor escalation flow
└── injection_guard.py # Prompt injection detection
supabase/
├── config.toml # Supabase CLI config
└── migrations/ # Versioned SQL schema
.github/workflows/
└── notifications-cron.yml # Manual fallback (workflow_dispatch) — the real cron runs on the host
deploy/
├── cron-notifications # File for /etc/cron.d/whatsapp-bot-notifications
└── RUNBOOK.md # Operations, deploy, secret rotation, domain migration
Dockerfile # Multi-stage (uv builder → slim runtime)
docker-compose.yml # app + caddy services
Caddyfile # Reverse proxy + automatic TLS
- Python 3.12 (see
requires-pythonin pyproject.toml) - A Supabase account with a project created
- A Meta for Developers app with WhatsApp Business configured
- A Google account with a GCP project and OAuth credentials
- An OpenAI API key with access to GPT-4o-mini
# 1. Create a virtual environment and install dependencies (recommended: uv)
uv sync
# 2. Create a .env file with the required variables (see the next section)
# 3. Apply the Supabase migrations to the remote project
npx supabase@latest login
npx supabase@latest link --project-ref <PROJECT_REF>
npx supabase@latest db push
# 4. Start the server
uv run fastapi devBy default it listens on http://localhost:8000. To expose the webhook to Meta during development, using a tunnel (ngrok, cloudflared) is recommended.
All are read from .env in development. Defined in app/config.py.
| Variable | Required | Description |
|---|---|---|
DATABASE_URL |
Yes | URL of the Supabase connection pooler (port 6543, not the direct 5432) |
WA_ACCESS_TOKEN |
Yes | Permanent Meta Business token for sending messages |
WA_PHONE_NUMBER_ID |
Yes | ID of the WhatsApp Business number |
WA_WEBHOOK_VERIFY_TOKEN |
Yes | Arbitrary token configured in Meta to verify the webhook |
WA_META_APP_SECRET |
Yes | Meta app secret, used to validate HMAC signatures on each request |
OPENAI_API_KEY |
Yes | API key with access to GPT-4o-mini |
GOOGLE_SERVICE_ACCOUNT_JSON |
Yes | Service Account JSON (on a single line, wrapped in single quotes) |
DRIVE_ROOT_FOLDER_ID |
Yes | ID of the root folder in the Shared Drive where receipts are archived |
LLM_MODEL |
No | OpenAI model for intent classification and receipt validation (default gpt-4o-mini) |
APP_NAME |
No | App name used for logging |
ENVIRONMENT |
No | development / staging / production |
LOG_LEVEL |
No | INFO by default |
BUSINESS_TIMEZONE |
No | Business timezone (default America/Argentina/Buenos_Aires) |
| Variable | Required | Description |
|---|---|---|
NOTIFICATIONS_INTERNAL_SECRET |
Yes | X-Internal-Secret header required by /internal/* (cron auth) |
WA_UPCOMING_TEMPLATE_NAME |
Yes | Name of the Meta-approved template for upcoming installments |
WA_OVERDUE_TEMPLATE_NAME |
Yes | Name of the Meta-approved template for overdue installments |
WA_TEMPLATE_LOCALE |
Yes | Template locale (e.g. es_AR) |
NOTIFICATIONS_UPCOMING_DAYS |
No | Advance-notice offsets in days, comma-separated (default 7,2) |
NOTIFICATIONS_OVERDUE_DAYS |
No | Post-due-date offsets for Phase 1 (default 1,3,7) |
NOTIFICATIONS_OVERDUE_WEEKLY_DAY |
No | Weekly day for Phase 2 (0=Monday, default 0) |
NOTIFICATIONS_QUIET_DAYS |
No | Days with no sending, comma-separated (default 5,6 — Saturday/Sunday) |
NOTIFICATIONS_DAILY_CAP |
No | Maximum sends per drain run (default 500) |
NOTIFICATIONS_TEST_RECIPIENT |
No | Phone number used by /internal/notifications/test when none is passed |
| Variable | Required | Description |
|---|---|---|
ESCALATION_ADVISOR_PHONE |
No | Phone number of the advisor who receives the escalation notification. Empty → the mailto fallback is used |
WA_ESCALACION_ASESOR_TEMPLATE_NAME |
No | Meta-approved template to notify the advisor (vars: nombre, telefono, motivo, resumen). Default hello_world → falls back to mailto until the real template is configured |
Note: several variables have legacy aliases (e.g.
WHATSAPP_TOKENinstead ofWA_ACCESS_TOKEN). See app/config.py for the complete list.
A ready-to-fill production template lives in deploy/env.production.example (key names with empty values).
Versioned schema in supabase/migrations/. Main tables:
| Table | Purpose |
|---|---|
clientes |
Trust holders (name, phone, trust, activo, opt-out) |
cuotas |
Monthly installments per client (amount, fecha_vencimiento, estado, periodo_label) |
pagos_reportados |
Uploaded receipts (wamid, Drive URL, status_verificacion, monto_pagado) |
user_states |
Conversational FSM state per phone number |
notificaciones |
Notification outbox (pending → sending → sent/failed/delivered/read) |
pago_aplicaciones |
Ledger of each payment → installment allocation (audit + trigger idempotency) |
creditos_cliente |
Surplus from approved payments, applied manually by the agent |
feriados |
Holidays for the year, used by the cron to shift sends to the previous business day |
The verification states of a payment are: pendiente_revision, aprobado, rechazado, duplicado.
The states of a notification are: pending, sending, sent, delivered, read, failed, skipped, escalated. The sent → delivered → read transition is driven by Meta's status webhook via reconcile_status.
When the bot persists a receipt in pagos_reportados, it stores in monto_pagado the amount the OCR (OpenAI vision) extracts from the PDF/image. pagos_reportados.cuota_id remains an informational pointer (traceability); application is NOT based on it, but on a FIFO walk over all of the client's installments.
The trigger trg_apply_pago_on_approval (AFTER INSERT OR DELETE OR UPDATE OF status_verificacion, monto_pagado) runs fn_apply_pago_on_approval, which acts as a dispatcher:
- New approval (transition to
aprobadoon INSERT or UPDATE) →fn_aplicar_pago(pago_id, notify := true). - Reversal, amount correction,
cliente_idchange, or DELETE of a payment that wasaprobado→fn_replay_cliente(cliente_id)(full recalculation).
fn_aplicar_pago(p_pago_id, p_notify, p_tolerancia := 1) — applies an approved payment:
- Locks (
FOR UPDATE) the client's unsettled installments (pendiente | parcial | vencido), ordered byfecha_vencimiento ASC, id ASC. - Assigns
monto_pagadoto the remaining balance of each installment in that order and records each allocation inpago_aplicaciones. Progress accumulates incuotas.monto_pagado_acumulado. Each touched installment becomesparcial, orpagadoif the remaining shortfall is<= p_tolerancia($1 by default).⚠️ The tolerance decouples state and ledger: an installment can end uppagadowithmonto_pagado_acumulado < monto(shortfall ≤ $1, receipt rounding). Queries that detect debt must filter by state (<> 'pagado'), not only by the ledger (monto > monto_pagado_acumulado), or phantom cents-of-debt would reappear.
- Any surplus (> $0.005), once all installments are settled, is inserted into
creditos_clientewithapplied_at = NULL. Credits are not auto-consumed: applying them is a manual action by the agent (seedeploy/RUNBOOK.md). - If
p_notify(only on new approvals, not on replay), it enqueues apago_aprobadonotification. It is idempotent: if the payment already has rows inpago_aplicaciones, it returns without doing anything.
fn_replay_cliente(p_cliente_id) — recalculates from scratch: resets the client's installments (monto_pagado_acumulado = 0, estado = 'pendiente'), purges its pago_aplicaciones and the auto-generated creditos_cliente (preserving the manual ones, with source_pago_id IS NULL), and reapplies all of its aprobado payments in created_at ASC order with notify := false. It does not touch notificaciones.
⚠️ The reset forcesestado = 'pendiente', assuming every unsettled installment starts that way. If in the future something marks an installment asvencidooren_disputa, a replay of that client would silently revert it to'pendiente'. Today it is harmless (no path writes those states), but keep it in mind before introducing automatic disputes/overdue transitions.
If monto_pagado is NULL or <= 0 (OCR failed), fn_aplicar_pago is a no-op. Implementation of the three functions and the pago_aplicaciones table: supabase/migrations/20260601000000_pago_aplicaciones_ledger_and_replay.sql.
# Create a new migration
npx supabase@latest migration new <name>
# Apply to the remote
npx supabase@latest db push
⚠️ The repo is NOT the source of truth for the live schema. Historically several changes were applied by hand (Supabase SQL editor), so the history ofsupabase_migrations.schema_migrationsin the database does not match the files in supabase/migrations/ — missing, among others, the migrations for thev_estado_de_cuentaview. Before adb pushor adb resetagainst the production database, check for drift; rebuilding from the repo does not reproduce the current state. This repo's migrations are written to be idempotent (CREATE OR REPLACE,IF NOT EXISTS) precisely so that documenting the live state without re-applying it is safe.
The backend connects with the Supabase connection pooler credential (see DATABASE_URL). Table access bypasses RLS by convention — it is a trusted backend.
| Method | Path | Auth | Description |
|---|---|---|---|
| GET/HEAD | /health |
Public | Health check ({"status": "ok"}) |
| GET | /webhook/whatsapp |
Verify token | Webhook verification by Meta (challenge/response) |
| POST | /webhook/whatsapp |
HMAC | Ingestion of WhatsApp messages and events |
| POST | /internal/notifications/run |
X-Internal-Secret |
Enqueue + drain of the notifications outbox |
| POST | /internal/notifications/test |
X-Internal-Secret |
Trigger a template to a specific number (debugging) |
The cron runs on the Lightsail VM via /etc/cron.d/whatsapp-bot-notifications (template file in deploy/cron-notifications), Monday to Friday at 12:00 UTC (09:00 ART):
0 12 * * 1-5 ubuntu curl -fsS -X POST http://localhost:8000/internal/notifications/run \
-H "X-Internal-Secret: $(cat /opt/whatsapp-bot/.cron-secret)" \
>> /var/log/whatsapp-bot-cron.log 2>&1The secret is read from /opt/whatsapp-bot/.cron-secret (chmod 600) so it is not exposed in ps. Logs are rotated via /etc/logrotate.d/whatsapp-bot (14 days, compress).
The workflow .github/workflows/notifications-cron.yml is left disabled (no schedule:) and is kept only as a manual fallback (workflow_dispatch) in case the VM is down. Secrets required in the repo for that fallback: APP_URL and NOTIFICATIONS_INTERNAL_SECRET.
The endpoint response includes:
enqueued_upcoming/enqueued_overdue/enqueued_overdue_weekly— how many notifications were added to the outbox.drained— how many were sent / failed / skipped in this run.opted_out_with_pending— how many active clients with pending installments are silenced (compliance receipt).failures_today— list of the day'sfailedentries with client, kind, and Meta error — useful for diagnosis (codes190,131026, etc.).
Defined in app/core/conversation.py. FSM states:
GREETING→ first contact; shows the interactive menu and moves toAWAIT_INTENT.AWAIT_INTENT→ waits for user input; resolves intent via digit shortcut, button reply, keyword fast-path, or LLM.AWAIT_INVOICE_UPLOAD→ waits for a file (PDF/image) of the receipt.
The priority order for classifying user text:
- Digit shortcut (
1,2,3) — from the legacy menu. - Button reply (from the interactive menu buttons or templates).
- Keyword fast-path — opt-out (
BAJA,PARAR,STOP,CANCELAR,NO MOLESTAR,DEJAR DE RECIBIR) and opt-in (REACTIVAR,VOLVER,SUSCRIBIR,START). - LLM (GPT-4o-mini) — free-text classification with a confidence threshold.
Recognized intents (see app/services/intent_classifier.py):
STATUS— check account status (runs an immediate query).AWAIT_INVOICE— report a payment (transitions toAWAIT_INVOICE_UPLOAD, after a duplicate check).ESCALATE— talk to a human advisor.OPT_OUT/OPT_IN— silence/reactivate automatic notifications (compliance with WhatsApp Business Policy).GREETING/UNKNOWN— fall back to the menu.
The main menu uses WhatsApp interactive buttons with three options: "Estado de tu cuenta", "Subir un comprobante", "Hablar con un asesor". Internally it is routed via a sentinel (<<INTERACTIVE_MENU>>) that the router detects and replaces with the sending of an interactive-type payload. The buttons return button_reply.id (e.g. STATUS), which maps directly to the corresponding intent.
Before transitioning to AWAIT_INVOICE_UPLOAD, the bot checks whether the client already has a payment in pendiente_revision for the current month's installment. If so, it informs the client and blocks the send. Additionally, at the webhook level, the wamid is used for idempotency (Meta may retry deliveries).
When a client sends an opt-out keyword, clientes.notifications_opted_out = TRUE is set with notifications_opted_out_at = NOW(). The client stays active (activo = TRUE) — they can start conversations, check status, and upload receipts — but are excluded from the cron enqueues. To receive notifications again, they simply send an opt-in keyword.
Implementation: app/services/notifications.py. Pattern: transactional outbox.
- Phase 1 — Upcoming: advance notices before the due date (configurable offsets, default T-7 and T-2 business days).
- Phase 1 — Overdue: post-due-date notices, anchored to the oldest pending installment (default T+1, T+3, T+7).
- Phase 2 — Weekly: once the Phase 1 offsets are exhausted, a weekly reminder on the configured day (default Monday).
- Approved payment: when an agent changes
pagos_reportados.status_verificaciontoaprobado, a DB trigger enqueues akind='pago_aprobado'notification with a JSONB snapshot of the outcome (settled installments, surplus tocreditos_cliente). It is delivered via theWA_PAGO_APROBADO_TEMPLATE_NAMEtemplate (example_pago_aprobado_v1) on the next cron run. Idempotent: a singlepago_aprobadoperpagos_reportados.id.
- Daily idempotency via the
ON CONFLICT DO NOTHINGconstraint on(cliente_id, cuota_id, kind, scheduled_for). - Safe concurrency via
FOR UPDATE SKIP LOCKEDwhen claiming the batch. - Retry with cap (
MAX_ATTEMPTS=3); after that the row ends upfailed. - Sweep of stale
sending(>10 min) to recover from mid-drain crashes. - Bidirectional reconciliation: Meta's status webhook advances
sent → delivered → reador marksfailed, with anti-downgrade ranking. - Business days + holidays: if the target falls on a "quiet" day (default Sat/Sun) or on a
feriadosentry, it is shifted to the previous business day withprevious_business_day. - Filter for payments in review: installments with
pagos_reportadosin statependiente_revisionoraprobadoare excluded from the candidate set (so it doesn't bother someone who already paid). - Opt-out filter: clients with
notifications_opted_out = TRUEare left out of the enqueue.
Each cron run returns a JSON with counts per kind, a list of the day's failures, and a count of opted-out clients with pending installments. Useful for diagnosing problems without querying the DB.
- Create an app on Meta for Developers.
- Add the WhatsApp product and configure a number.
- In Configuration → Webhooks, set:
- Callback URL:
https://<your-domain>/webhook/whatsapp - Verify token: the value of
WA_WEBHOOK_VERIFY_TOKEN
- Callback URL:
- Subscribe to the messages and message_template_status_update fields (the latter optional for reconcile).
- Generate a permanent token (System User) and load it into
WA_ACCESS_TOKEN. Temporary tokens expire after 24h. - Copy the App Secret to
WA_META_APP_SECRET. - Create and approve the notification templates in Meta Business Manager (the names go in
WA_UPCOMING_TEMPLATE_NAME,WA_OVERDUE_TEMPLATE_NAME,WA_UPCOMING_SALDO_REMANENTE_TEMPLATE_NAME,WA_OVERDUE_INTERES_AVISO_TEMPLATE_NAME, andWA_PAGO_APROBADO_TEMPLATE_NAME). The reminder templates include 3 Quick Reply buttons with IDsSTATUS,AWAIT_INVOICE,ESCALATE.
The bot uses a Service Account from the client's Google Workspace, operating on a Shared Drive of which the SA is a member. The files remain owned by the client's Workspace, not by a personal account. Setup flow:
- In Google Cloud Console (the client's project), enable the Google Drive API.
- Create a Service Account and download the JSON key.
- In Google Drive, create (or use) a Shared Drive and add the SA email as a member with "Content manager" permission.
- Copy the ID of the Shared Drive (or of a folder within it) to
DRIVE_ROOT_FOLDER_ID. - Paste the JSON contents into
GOOGLE_SERVICE_ACCOUNT_JSONon a single line, wrapped in single quotes.
API calls are made with supportsAllDrives=True / includeItemsFromAllDrives=True to operate on Shared Drives.
The folder structure created in Drive is: {Year}/{Month}/{fideicomiso_id}_{cliente_nombre}/{date}.pdf.
GPT-4o-mini is used for two functions:
- Intent classification (app/services/intent_classifier.py) — free user text → intent enum with a confidence score.
- Receipt validation (app/services/document_ai.py) — analyzes the PDF/image, determines whether it is a valid bank receipt, and extracts amount, date, sender, receiver, and transaction ID.
Only PostgreSQL is used (no auth, no storage). Connect with the connection pooler (port 6543) from serverless/edge backends.
- HMAC signature (
X-Hub-Signature-256) validated on every webhook request usingWA_META_APP_SECRET. - Gating by active client — the bot resolves the incoming number against
clientesand rejects if it does not exist oractivo = FALSE. - Protected internal endpoints via the
X-Internal-Secretheader. - Prompt injection guard (app/services/injection_guard.py) — heuristic check before the LLM.
- Per-client lock on
user_states(SELECT ... FOR UPDATE) — guarantees FSM atomicity against concurrent messages from the same number. - Idempotency by
wamidinpagos_reportados— prevents duplicates on Meta retries. - Structured JSON logs — events serialized with internal IDs for traceability.
Production runs on AWS Lightsail (Ubuntu 22.04, $10/mo plan: 1 GB RAM, 2 vCPU burst, 40 GB SSD) in us-east-1, with an attached static IP and <ip>.nip.io as the webhook FQDN (no dedicated domain registered, a deliberate decision at project close — see deploy/RUNBOOK.md for the procedure to migrate to a dedicated domain).
The runtime is packaged as Docker Compose (2 services):
app— multi-stage image built from theDockerfile(python:3.12-slim+uv+ uvicorn). Binds only to127.0.0.1:8000so the host cron can call it directly, without going through TLS.caddy—caddy:2-alpine, exposes:80and:443, issues and renews Let's Encrypt certs automatically (HTTP-01 challenge). Config inCaddyfile.
Deploying changes:
ssh ubuntu@<static-ip>
cd /opt/whatsapp-bot
git pull
docker compose up -d --buildHardening applied on the VM: unattended-upgrades (automatic security patches + reboot at 03:00), logrotate for the cron log. No automatic snapshots, by project decision.
Full operations, secret rotation, snapshot restore, migration to a dedicated domain, and manual payment approval: see deploy/RUNBOOK.md.
- Annual holiday seed — a process to load holidays for future years (currently only 2026).
- Migration to a dedicated domain — optional, once a domain is registered. Procedure documented in
deploy/RUNBOOK.md. - Admin dashboard — currently payment approval and
creditos_clienteapplication are done with direct SQL.