Add abuse protection (rate limiting) to /api/query

[leekahung]

Goal

Protect the expensive /api/query endpoint from scripted abuse / cost-draining. Every call to /api/query fans out to Vertex AI RAG + Gemini, so unthrottled traffic is a direct cost and availability risk.

Current state (June 2026)

• Only two routes exist: /api/query (chat) and /api/feedback. The endpoints originally named in this issue (/api/history, /api/clear-session, /api/init, /api/citation) were removed when session state moved client-side — the frontend now posts the full message history per request, so there's no server-side session to lock down.
• Already in place: strict CORS origin validation and flask_limiter.
• The gap: the rate limit is currently applied only to /api/feedback. The costly /api/query endpoint has no rate limit and no bot protection.

Scope

1. Apply a rate limit to /api/query (primary fix — flask_limiter is already wired up).
2. (Optional follow-up) Add bot mitigation — Cloudflare Turnstile (fully free) or hCaptcha (free tier), via a token verified server-side. Only pursue if the rate limiter proves insufficient against distributed/bot abuse.

Explicitly out of scope

User authentication (JWT / OAuth / login sessions). This is an anonymous public legal-aid tool for tenants in crisis; requiring login adds friction for exactly the users we want to reach. Abuse protection, not identity, is the actual need.

Original issue text (stale — kept for history)

Before implementing feedback, we'll need to lock down our API endpoints with some authentication solution to prevent potential exploits.

One potential solution would be with tokens.

Why stale: This was written against a session-based API with five endpoints, four of which no longer exist, and framed the solution as authentication. The architecture is now stateless (client holds message history) and the realistic threat is abuse of the costly LLM endpoint — addressed by rate limiting, not user auth.

[yangm2]

@claude please describe at least 3 potential implementations of this feature, compare and contrast the trade-offs of the implementations and which implementations are conventional vs. unconventional in the industry

[claude]

Claude finished @yangm2's task —— View job

---

Authentication Implementation Analysis

I've analyzed the current codebase and will provide 3+ potential authentication implementations with trade-offs and industry conventions.

Todo List:

• Read repository setup and guidelines
• Examine current API structure and tech stack
• Research and document 3+ authentication approaches
• Compare trade-offs and industry conventions
• Provide recommendations

Current System Overview

Tech Stack: Flask backend with React frontend
Current API Endpoints: /api/history, /api/clear-session, /api/init, /api/query, /api/citation
Current Security: Basic Flask sessions only, no authentication

---

Authentication Implementation Options

1. JWT (JSON Web Tokens) - CONVENTIONAL

Implementation: Issue tokens on login, validate on each request via middleware

# Middleware example
from functools import wraps
import jwt

def token_required(f):
    @wraps(f)
    def decorated(*args, **kwargs):
        token = request.headers.get('Authorization')
        if not token:
            return jsonify({'error': 'Token missing'}), 401
        try:
            jwt.decode(token.replace('Bearer ', ''), app.secret_key, algorithms=['HS256'])
        except jwt.InvalidTokenError:
            return jsonify({'error': 'Token invalid'}), 401
        return f(*args, **kwargs)
    return decorated

Trade-offs:

• ✅ Pros: Stateless, scalable, industry standard, works well with SPAs
• ✅ Security: Can include expiration, claims, refresh tokens
• ❌ Cons: Tokens can't be revoked easily, complexity in refresh logic
• ❌ Overhead: JWT parsing on each request

Industry Status: Very conventional, widely adopted for API authentication

---

2. API Key Authentication - CONVENTIONAL

Implementation: Generate API keys for users/clients, validate via database lookup

@app.before_request
def validate_api_key():
    if request.endpoint in ['api.history', 'api.query']:
        api_key = request.headers.get('X-API-Key')
        if not api_key or not is_valid_api_key(api_key):
            return jsonify({'error': 'Invalid API key'}), 401

Trade-offs:

• ✅ Pros: Simple implementation, easy revocation, good for service-to-service
• ✅ Performance: Fast database lookup
• ❌ Cons: Not ideal for user-facing apps, key management complexity
• ❌ Security: Keys can be logged/cached accidentally

Industry Status: Conventional for APIs, less common for web applications

---

3. Session-Based with Enhanced Security - CONVENTIONAL

Implementation: Extend existing Flask sessions with proper user authentication

from flask_session import Session
import valkey  # Already in dependencies

app.config['SESSION_TYPE'] = 'redis'  # Use existing Valkey
app.config['SESSION_USE_SIGNER'] = True
app.config['SESSION_KEY_PREFIX'] = 'tenant_auth:'

def login_required(f):
    @wraps(f)
    def decorated(*args, **kwargs):
        if 'user_id' not in session:
            return jsonify({'error': 'Login required'}), 401
        return f(*args, **kwargs)
    return decorated

Trade-offs:

• ✅ Pros: Leverages existing Redis/Valkey infrastructure, server-side control
• ✅ Security: Easy session invalidation, CSRF protection built-in
• ❌ Cons: Requires server-side storage, not ideal for mobile/API-first
• ❌ Scalability: Session stickiness issues in multi-server setups

Industry Status: Very conventional for web applications

---

4. OAuth 2.0 / OpenID Connect - CONVENTIONAL

Implementation: Integrate with providers (Google, GitHub, Auth0)

from authlib.integrations.flask_client import OAuth

oauth = OAuth(app)
google = oauth.register(
    name='google',
    client_id=os.getenv('GOOGLE_CLIENT_ID'),
    client_secret=os.getenv('GOOGLE_CLIENT_SECRET'),
    server_metadata_url='https://accounts.google.com/.well-known/openid-configuration'
)

Trade-offs:

• ✅ Pros: Offloads auth complexity, user convenience, enterprise-ready
• ✅ Security: Proven implementations, MFA support
• ❌ Cons: External dependencies, privacy concerns, integration complexity
• ❌ Cost: Potential costs for enterprise providers

Industry Status: Very conventional, especially for SaaS applications

---

5. Rate-Limited IP Authentication - UNCONVENTIONAL

Implementation: IP-based access control with rate limiting

from flask_limiter import Limiter
from flask_limiter.util import get_remote_address

limiter = Limiter(
    app,
    key_func=get_remote_address,
    default_limits=["100 per hour"]
)

@app.route('/api/query')
@limiter.limit("10 per minute")
def query():
    # Existing logic

Trade-offs:

• ✅ Pros: Simple, no user management, immediate implementation
• ✅ Protection: Prevents abuse without full auth system
• ❌ Cons: Not true authentication, IP sharing issues, VPN bypass
• ❌ User Experience: Anonymous users, no personalization

Industry Status: Unconventional as primary auth, common for API protection

---

Recommendations

For TenantFirstAid specifically:

1. Short-term (Immediate): Implement rate-limited IP authentication (Idea: A Vector database #5) to prevent immediate exploits while planning proper auth

2. Medium-term (Recommended): Enhanced Session-Based Authentication (Frontend design tweaks #3)

   • Leverages existing Valkey infrastructure
   • Fits web application model
   • Maintains simplicity
   • Good fit for legal advice chatbot where user sessions matter

3. Long-term consideration: JWT tokens (Format the output #1) if you need:

   • Mobile app support
   • API-first architecture
   • Multi-service authentication

Implementation Priority:

1. Rate limiting (immediate protection)
2. User registration/login system
3. Session-based auth with Valkey storage
4. Admin panel for user management

The session-based approach aligns best with your current architecture and use case, while providing the security you need without over-engineering.

[SuhrudhC]

Hi @scheidtdav, I would love to work on this issue. I'm a rising college freshman and a student in CodePath's AI 301 course. After reading over this issue, I've found that it aligns well with the backend issues I've worked on in the past. I'd love to read through all the documentation and understand the program further, if still necessary. Let me know if this issue is still unresolved!

[leekahung]

Given the current state of the codebase and the changes over the past year, this issue requires re-scoping. Updating the issue description.

[SuhrudhC]

Hey, following up on my comment from a few weeks back, I went ahead and worked through this.

Turned out the fix itself was pretty small once I traced it down: flask_limiter was already set up in app.py and already being used on /api/feedback, it just was never attached to /api/query. So the actual change is just wrapping the chat view with limiter.limit() the same way feedback already does it, defaulting to 10 requests per minute per IP and configurable via an env var if that number needs adjusting later.

Before writing any fix code, I put together a small script that reproduces the bug by bursting both endpoints and comparing the results, so there's a clear before/after. I also wrote a pretty thorough set of tests beyond just "does it return 429," checked the exact boundary, that different IPs get independent quotas, that the limit window resets correctly, and that this doesn't accidentally affect the feedback endpoint's existing limit.

PR is up here: #376. Would appreciate a look whenever someone has time, and if I'm missing anything about how this should behave in production (especially around how the rate limiter sees client IPs behind the proxy), let me know. I flagged that as an open question in the PR itself.