An autonomous "Computational Scientist" that watches videos of physical phenomena, reasons about the underlying physics, writes simulation code, and discovers discrepancies in real-time.
Physics education is broken. Students memorize formulas (
- Textbooks are static: They describe ideal vacuums, not reality.
- Simulations are pre-canned: You tweak a slider; you don't build the model.
- AI usually just chats: LLMs can explain physics, but they "hallucinate" math and can't see the world accurately.
Q.E.D. solves this by closing the loop between perception, reasoning, and simulation. It doesn't just "watch" reality; it understands it well enough to rebuild it.
Real Video Uploaded → Gemini 3 Eyes (Analysis) → Physics Parameters Extracted → Gemini 3 Brain (Reasoning) → Manim Code Generated → Simulation Rendered → Reality vs. Sim Comparison
Total Time: <60 seconds
| Feature | Description | The "Wow" Factor |
|---|---|---|
| Multimodal Vision | Extracts variables (length, mass, angle) from raw video | No manual data entry |
| Deep Think Reasoning | Derives governing equations from first principles | Uses LaTeX to "write" math |
| Digital Twin | Generates Python (Manim) code to simulate the event | Visual proof of understanding |
| Newton Moment | Detects when simulation |
Self-correcting AI |
| Voice Narration | Real-time audio commentary of the thought process | Feels like a human tutor |
Q.E.D.'s most advanced feature is its ability to fail and learn.
-
Hypothesis: "This is a simple pendulum.
$T \approx 2\pi\sqrt{L/g}$ ." - Simulation: Runs the ideal model.
- Observation: "Wait. The real pendulum is slowing down. The simulation runs forever."
-
Correction: "Discrepancy detected. Hypothesis revised: Air resistance is non-negligible. Adding damping term
$-bv$ ." - Re-Simulation: The new model matches reality perfectly.
┌─────────────────────────────────────┐
│ Q.E.D. SYSTEM │
└─────────────────────────────────────┘
│
┌────────────────────────────────┼────────────────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ PERCEPTION │ │ COGNITION │ │ RENDERING │
│ │ │ │ │ │
│ • Video In │──────────────│ • Gemini 3 Pro │──────────────│ • Manim Engine │
│ • OpenCV │ Params │ • Deep Think │ Code │ • WebGL Canvas │
│ • Frame Ext. │──────────────│ • SymPy │──────────────│ • React UI │
│ │ │ │ │ │
└───────────────┘ └─────────────────┘ └─────────────────┘
- Input: User uploads video (e.g., specific pendulum swing).
- Vision: Backend extracts frames. Gemini Vision identifies object & estimates parameters (Length=0.32m).
- Derivation: Reasoning Engine prompts Gemini to derive the Equation of Motion.
- Codegen: System generates a standard
manimPython script with these variables. - Execution: Sandbox runs the Python code to generate a video/animation.
- Comparison: Frontend plays Real Video (left) vs. Simulation (right) in sync.
| Choice | Why We Chose This |
|---|---|
| Next.js + Tailwind | For the "Deep Void" aesthetic (Glassmorphism, Grid backgrounds). Performance is key for video sync. |
| FastAPI | Async handling of long-running AI jobs. Native support for Pydantic validation. |
| Gemini 3 Pro | The only model with the long-context multimodal reasoning needed to correlate video frames with complex math. |
| Manim | The gold standard for programmatic math animation (created by 3Blue1Brown). |
| Docker | Sandboxed execution of AI-generated code. Preventing os.system("rm -rf /") is a priority. |
- Node.js 18+
- Python 3.10+
- FFmpeg (for Manim)
- Gemini API Key
# Clone repository
git clone https://github.com/yourusername/qed.git
cd qed
# 1. Setup Backend
cd backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env # Add GEMINI_API_KEY
# 2. Setup Frontend
cd ../frontend
npm install# Terminal 1: Backend
cd backend
uvicorn app.main:app --reload
# Terminal 2: Frontend
cd frontend
npm run devOpen http://localhost:3000 and enter the lab.
qed/
├── backend/ # FastAPI Brain
│ ├── app/
│ │ ├── gemini/ # AI Client
│ │ ├── services/ # Reasoning & Sandbox logic
│ │ └── main.py # Entry point
│ ├── templates/ # Manim script templates
│ └── requirements.txt
│
├── frontend/ # Next.js Laboratory
│ ├── app/ # Pages (Upload, Analysis)
│ ├── components/ui/ # Glassmorphic UI Kit
│ └── lib/ # Utilities
│
└── simulation-engine/ # Dockerized Manim runner
AI is great at writing code, but bad at physics precision. It would often guess
Generating a video takes time. Users don't want to wait 60 seconds. Solution: We built a Digital Twin Canvas fallback. While the high-quality Manim video renders in the background, a lightweight WebGL simulation runs instantly on the frontend, giving immediate feedback.
- Challenge Mode: Gamified physics ("Can you predict where the ball lands?").
- AR Integration: Project the simulation overlay onto the real world using mobile camera.
- More Phenomena: Support for springs, fluid dynamics, and orbital mechanics.
Built for the Gemini 3 Hackathon
"Nature is written in the language of mathematics." - Galileo