Skip to content

WEATHER_SUBSCRIPTION.md

Latest commit

 

History

History
434 lines (334 loc) · 13.8 KB

WEATHER_SUBSCRIPTION.md

File metadata and controls

434 lines (334 loc) · 13.8 KB
Raw

Weather Subscription Feature Progress

Overview

Weather subscription allows farmers to opt-in to receive daily weather forecast messages for their administrative area via WhatsApp.

Current Status: Phase 3 In Progress

Phase 1: Message Generation (DONE)

Implemented the ability to fetch weather data and generate farmer-friendly broadcast messages.

Key Files

File Purpose
services/weather_broadcast_service.py Core service for weather data retrieval and message generation
services/weather_subscription_service.py Manages subscription preferences (subscribe/decline)
routers/weather.py Admin test endpoint for message generation
schemas/weather.py Request schema with location and language enum
templates/weather_broadcast.txt Prompt template for OpenAI message generation
tests/test_weather_broadcast_service.py Service tests (20 tests)
tests/test_weather_router.py Router tests (9 tests)

Configuration

Environment variables (in .env):

OPENWEATHER=your_openweather_api_key

Config.json:

{
  "weather": {
    "broadcast_enabled": true,
    "api_version": "3.0",
    "intent_keywords": [
      "weather",
      "forecast",
      "weather updates",
      "hali ya hewa",
      "hali ya anga"
    ]
  }
}

API version options:

  • "3.0" - Uses OneCall API 3.0 with lat/lon coordinates (requires paid subscription). Falls back to 2.5 if coordinates not available or API fails.
  • "2.5" - Uses location-based API (free tier). Default if not specified.

How It Works

  1. WeatherBroadcastService.get_weather_data(location, lat, lon) - Fetches weather data using configured API version:
    • API 3.0: Uses get_current_raw(lat, lon) with OneCall API
    • API 2.5: Uses get_forecast_raw(location) with location-based API
    • Falls back to 2.5 if coordinates not provided or 3.0 fails
  2. WeatherBroadcastService.generate_message(location, language, weather_data, farmer_crop) - Uses OpenAI to generate a farmer-friendly message based on the prompt template, with crop-specific suggestions if farmer_crop is provided

Test Endpoint

POST /api/admin/weather/test-message
Authorization: Bearer <admin_token>
Content-Type: application/json

{
  "location": "Kiru, Mathioya, Murang'a",
  "lat": -0.85,          // Optional: for OneCall 3.0 API
  "lon": 36.95,          // Optional: for OneCall 3.0 API
  "crop_type": "Avocado", // Optional: for crop-specific advice
  "language": "en"       // or "sw" for Swahili
}

Returns plain text message with crop-specific weather suggestions if crop_type is provided.

Message Format

The generated message follows this structure:

  1. Short title
  2. Simple weather summary (2-3 lines) with temperature range and rainfall
  3. "What to do today" section with bullet points (spraying, field work, watering)
  4. One short reminder sentence

Max 1000 characters, WhatsApp-friendly formatting, no technical terms.

Phase 2: Broadcast to Subscribers (DONE)

Implemented daily weather broadcasts to subscribed farmers with template confirmation workflow.

Key Files

File Purpose
models/weather_broadcast.py WeatherBroadcast and WeatherBroadcastRecipient models
tasks/weather_tasks.py Celery tasks for broadcasting
alembic/versions/..._create_weather_broadcast_tables.py Database migration
tests/test_weather_tasks.py Task tests (16 tests)

Database Schema

WeatherBroadcast (daily weather broadcast per area+crop)
├── administrative_id (FK)
├── crop_type (VARCHAR 100, indexed)
├── location_name
├── weather_data (JSON)
├── generated_message_en
├── generated_message_sw
├── status (pending, processing, completed, failed)
├── scheduled_at, started_at, completed_at

WeatherBroadcastRecipient (delivery tracking)
├── weather_broadcast_id (FK)
├── customer_id (FK)
├── status (DeliveryStatus enum)
├── confirm_message_sid (for webhook callback)
├── actual_message_sid
├── retry_count, error_message
├── sent_at, confirmed_at

Note: Broadcasts are grouped by (administrative_id, crop_type) to provide crop-specific weather advice. An index on (administrative_id, crop_type) supports efficient grouping.

Workflow

sequenceDiagram
    participant CB as Celery Beat
    participant WT as Weather Tasks
    participant OW as OpenWeatherMap
    participant AI as OpenAI
    participant WA as WhatsApp
    participant F as Farmer

    Note over CB: 6 AM UTC Daily
    CB->>WT: send_weather_broadcasts
    WT->>WT: Query subscribed farmers by (area, crop_type)
    WT->>WT: Create WeatherBroadcast records

    loop Per Area+Crop Group
        WT->>OW: Fetch weather data (using lat/lon if available)
        OW-->>WT: Raw weather data
        WT->>AI: Generate crop-specific message (EN/SW)
        AI-->>WT: Farmer-friendly message with crop advice
        WT->>WA: Send template to subscribers
        WA-->>F: "New weather update available"
    end

    F->>WA: Clicks "Yes" button
    WA->>WT: Webhook callback
    WT->>WA: Send actual weather message
    WA-->>F: Crop-specific weather forecast
Loading

  1. Daily Task (send_weather_broadcasts) runs at 6 AM UTC via Celery Beat

  2. Queries customers with weather_subscribed = True grouped by (administrative area, crop_type)

  3. Creates WeatherBroadcast record per area+crop combination

  4. Queues send_weather_templates task per area+crop

  5. Template Sending (send_weather_templates):

    • Fetches weather data via OpenWeatherMap (using lat/lon for OneCall 3.0 if configured)
    • Generates crop-specific messages in EN and SW via OpenAI
    • Filters subscribers by matching crop_type
    • Sends WhatsApp template (reuses broadcast template SID)
    • Creates WeatherBroadcastRecipient with confirm_message_sid

  6. User Confirmation:

    • User clicks "Yes" button (read_broadcast payload)
    • Webhook checks WeatherBroadcastRecipient first (priority over regular broadcasts)
    • Queues send_weather_message task

  7. Actual Message (send_weather_message):

    • Sends language-specific weather message
    • Creates Message record
    • Updates recipient with actual_message_sid

  8. Retry Mechanism (retry_failed_weather_broadcasts):

    • Runs every 5 minutes via Celery Beat
    • Retries failed recipients at intervals [5, 15, 60] minutes
    • Marks as UNDELIVERED after max retries

Celery Beat Schedule

# celery_app.py
"send-weather-broadcasts": {
    "task": "tasks.weather_tasks.send_weather_broadcasts",
    "schedule": crontab(hour=6, minute=0),  # 6 AM UTC daily
},
"retry-failed-weather-broadcasts": {
    "task": "tasks.weather_tasks.retry_failed_weather_broadcasts",
    "schedule": crontab(minute="*/5"),  # Every 5 minutes
},

Integration Points

  • Webhook (routers/whatsapp.py): Weather recipients checked before regular broadcasts
  • Template: Reuses existing broadcast template SID (language-aware)
  • Button Payload: Reuses read_broadcast payload

Phase 2.5: Weather Intent Detection (DONE)

Implemented on-demand weather message generation when farmers ask about weather.

Key Files

File Purpose
services/weather_intent_service.py Detects weather keywords and generates weather messages
tests/test_weather_intent.py Weather intent tests

How It Works

flowchart TD
    A[Farmer sends message] --> B{Contains weather keyword?}
    B -->|No| C[Continue normal flow]
    B -->|Yes| D{Onboarding complete?}
    D -->|No| C
    D -->|Yes| E{Has admin area?}
    E -->|No| C
    E -->|Yes| F{Has open ticket?}
    F -->|Yes| C
    F -->|No| G[Build location path]
    G --> H[Fetch weather from OpenWeatherMap]
    H --> I[Generate message via OpenAI]
    I --> J[Send weather message]
    J --> K{Already subscribed?}
    K -->|Yes| L[Done]
    K -->|No| M[Show subscription buttons]
    M --> L
Loading
  1. Farmer sends message containing weather keywords (e.g., "weather updates", "hali ya hewa")
  2. WeatherIntentService.has_weather_intent() detects keywords from config.json
  3. WeatherIntentService.can_handle() checks conditions:
    • Customer has completed onboarding
    • Customer has an administrative area assigned
    • No existing escalated ticket
  4. WeatherIntentService.handle_weather_intent():
    • Builds full location path from administrative hierarchy (Region, District, Ward)
    • Generates weather message via WeatherBroadcastService
    • Sends weather message to farmer
    • Shows subscription buttons if farmer is not already subscribed

Re-subscription Flow

Farmers who previously declined weather subscription can re-subscribe:

  1. Farmer messages "weather updates" (or any configured keyword)
  2. System sends current weather for their area
  3. System shows subscription buttons again
  4. Farmer can now subscribe by clicking "Yes"

This allows farmers to change their mind without needing admin intervention.

Phase 3: Admin Management (Partial)

Admin interface to manage weather broadcasts.

Implemented Features

  1. Manual Trigger Endpoint - Trigger weather broadcasts on-demand
POST /api/admin/weather/trigger-broadcast
Authorization: Bearer <admin_token>

Response (202 Accepted):
{
  "status": "queued",
  "task_id": "abc123-def456",
  "message": "Weather broadcast task queued successfully"
}

Remaining Features (TODO)

  1. View subscription statistics by area
  2. Preview message before sending
  3. View broadcast history/logs
  4. Enable/disable broadcasts per area

Subscription Flow

Implemented in weather_subscription_service.py and weather_intent_service.py:

flowchart TD
    A[Farmer completes onboarding] --> B{Has admin area?}
    B -->|No| C[No weather subscription offered]
    B -->|Yes| D[Show subscription question]
    D --> E{Farmer response}
    E -->|Yes| F[weather_subscribed = true]
    E -->|No| G[weather_subscribed = false]
    F --> H[Receive daily broadcasts]
    G --> I[No broadcasts]
    I --> J[Farmer messages 'weather updates']
    J --> K[Send current weather]
    K --> L[Show subscription buttons again]
    L --> M{Farmer response}
    M -->|Yes| F
    M -->|No| I
Loading

Initial Subscription (After Onboarding)

  1. After onboarding completion + admin area assignment
  2. System asks: "Would you like to receive daily weather updates for {area_name}?"
  3. Customer responds Yes/No via WhatsApp buttons
  4. Preference stored in customer.profile_data:
    • weather_subscription_asked: bool
    • weather_subscribed: bool | null

Re-subscription (After Decline)

Farmers who declined can subscribe later:

  1. Farmer sends message with weather keyword (e.g., "weather updates")
  2. System generates and sends current weather for their area
  3. System shows subscription buttons again
  4. Farmer clicks "Yes" to subscribe

Button payloads configured in config.py:

  • weather_yes_payload: "weather_yes"
  • weather_no_payload: "weather_no"

Dependencies

  • akvo-weather-info>=0.3.0 - Weather API wrapper (Google Weather)
  • OpenAI API - Message generation
  • Existing services: openai_service.py, whatsapp_service.py, broadcast_service.py

Changelog

v0.3.0 - Google Weather Migration (2026-03-25)

Migrated from OpenWeatherMap to Google Weather API for improved accuracy and reliability.

Breaking Changes

  • Environment variable renamed: OPENWEATHER → GOOGLEWEATHER
  • Library updated: akvo-weather-info>=0.2.0 → akvo-weather-info>=0.3.0
  • API version config removed: weather.api_version no longer used

Migration Guide

  1. Update environment variable in .env:

    - OPENWEATHER=your_openweather_api_key
+ GOOGLEWEATHER=your_google_weather_api_key

  2. Get Google Weather API key:

  3. Update docker-compose (if using custom compose file):

    environment:
-  - OPENWEATHER=${OPENWEATHER}
+  - GOOGLEWEATHER=${GOOGLEWEATHER}

  4. Remove deprecated config from config.json:

    "weather": {
  "broadcast_enabled": true,
  "intent_enabled": false,
-  "api_version": "3.0",
  "intent_keywords": [...]
}

  5. Restart services:

    ./dc.sh down && ./dc.sh up -d

Files Changed

File Change
backend/requirements.txt Updated akvo-weather-info>=0.3.0
backend/config.py Replaced openweather_api_key with google_weather_api_key, removed weather_api_version
backend/services/weather_broadcast_service.py Changed from OpenWeatherMapService to GoogleWeatherService
docker-compose.yml Replaced OPENWEATHER with GOOGLEWEATHER
backend/config.json Removed api_version
backend/config.template.json Removed api_version
.env.example Updated env var documentation

Crop-Specific Weather Broadcasts

Weather broadcasts are now tailored per crop type:

How It Works

  1. Subscribers are grouped by (administrative area + crop_type)
  2. Each group receives crop-specific weather advice
  3. The prompt template includes {{ farmer_crop }} placeholder
  4. OpenAI generates advice specific to each crop's needs

Example

For Avocado farmers in Kiru:

  • Message includes avocado-specific tips (e.g., "Avoid spraying fungicides today due to rain")
  • Temperature recommendations relevant to avocado cultivation
  • Irrigation advice based on soil moisture needs

For Maize farmers in the same area:

  • Different advice relevant to maize growing stage
  • Recommendations for maize-specific pests and diseases

Configuration

Customer's crop_type is stored in profile_data.crop_type (set during onboarding). If crop_type is not set, subscribers are grouped as "unknown" and receive generic weather advice.