AI Paper Grader

---

Demo

ai-grader-demo-compressed.mp4

Upload a student's answer sheet (PDF), get back an AI-generated score and structured feedback automatically.

Built with Go. Designed around a microservices architecture with async grading via Kafka and vision-based LLM evaluation.

---

How it works

flowchart LR
    Client -->|JWT| Gateway
    Gateway -->|proxy| Upload
    Gateway -->|proxy| Results

    Client -->|presigned PUT| S3
    Upload -->|verify upload + outbox event| Kafka

    Kafka -->|consume| Grader
    Grader -->|fetch PDF| S3
    Grader -->|render pages → images| LLM["OpenRouter LLM"]
    LLM -->|score + feedback| Grader
    Grader -->|write grade| DB[(Postgres)]

    Results -->|read| DB

Loading

1. Client asks the gateway, which proxies to Upload, for a short-lived authenticated presigned S3 upload URL
2. Client uploads the PDF directly to private S3, then asks the gateway to proxy verification and completion to Upload
3. Upload verifies file metadata and PDF magic bytes, then creates the submission and paper-uploaded outbox event in one transaction
4. Grader service picks it up, renders each PDF page to an image, and sends them to an OpenRouter vision LLM with the rubric
5. Score and structured feedback land in Postgres
6. Client polls /results to get the grade

---

Architecture

graph TD
    subgraph Services
        A[API Gateway :8080]
        B[Upload Service :8082]
        C[Grader Service]
        D[Results Service :8083]
    end

    subgraph Infra
        K[Kafka]
        P[(Postgres)]
        S[S3]
    end

    A --> B
    A --> D
    B --> K
    B --> P
    B --> S
    A --> P
    Client --> S
    K --> C
    C --> S
    C --> P
    D --> P

Loading

Service	Role
API Gateway	Auth, CORS, and Upload/Results proxy
Upload Service	Presigned-upload issuance, verified completion, transactional outbox, and outbox publishing
Grader Service	Kafka consumer — renders PDF pages, calls vision LLM, saves grade
Results Service	Read-only API for submission status and grades

---

Tech stack

• Go — all services
• PostgreSQL (pgx) — submissions, grades, outbox table
• Kafka — async decoupling between upload and grading
• AWS S3 — PDF storage
• OpenRouter — vision LLM (sends rendered page images + rubric)
• go-fitz — PDF → image rendering
• JWT — access + refresh token auth
• Docker Compose — local infra

---

Design decisions worth noting

Outbox pattern — instead of publishing directly to Kafka after a DB write (two-phase risk), the upload service writes an event row in the same transaction. A background publisher polls and forwards to Kafka. No lost events on crash.

Vision-based grading — PDF pages are rendered to PNG images and sent as base64 to the LLM. This handles handwritten answers, diagrams, and non-selectable text naturally.

Single gateway — the gateway is the public entry point and proxies authenticated Upload and Results requests. Both downstream services independently validate the forwarded JWT before using its user claim. This prevents direct-service callers from spoofing identity with X-User-ID.

---

Running locally

# 1. Start infra
docker compose up -d postgres zookeeper kafka

# 2. Migrate
docker exec -i ai-grader-db psql -U ai-grader -d ai_grader < migrations/001_schema.sql
docker exec -i ai-grader-db psql -U ai-grader -d ai_grader < migrations/002_upload_intents.sql

# 3. Copy and fill env
cp .env.example .env

# 4. Run services (separate terminals)
go run ./cmd/api
go run ./cmd/upload
go run ./cmd/grader
go run ./cmd/results

Required env vars: DATABASE_URL, JWT_SECRET, AWS_*, S3_BUCKET_NAME, OPENROUTER_API_KEY, GLOBAL_GRADING_RUBRIC

---

API

POST /auth/register
POST /auth/login
POST /auth/refresh

POST /uploads/presign     # JSON metadata → short-lived S3 PUT URL
POST /uploads/:id/complete # verify S3 object and create submission
GET  /results             # list user's submissions
GET  /results/:id         # single result with score + feedback

All /uploads/* and /results routes require Authorization: Bearer <token>.

S3 browser upload requirements

Keep the bucket private. Configure bucket CORS for the frontend origins to allow PUT with Content-Type and x-amz-* headers, and expose ETag. The gateway still verifies the object size and %PDF- signature before creating a submission. Add a lifecycle rule to expire abandoned uploads/ objects, and grant the gateway only PutObject, HeadObject, ranged GetObject, and DeleteObject permissions for that prefix.