Vyn

An AI-powered voice assistant with a FastAPI backend and a modern React (Vite + TypeScript + Tailwind) frontend. Speak or type commands like "open chrome and check gmail from wellfound" or "open spotify and play tum hi ho"—Vyn parses, executes, and speaks back results.

Features

• Natural language command parsing with chaining (e.g., "open chrome and check gmail...")
• Voice wake word and debounce to avoid mid-utterance execution ("Hey Vyn")
• Gmail search + UI navigation via URL (thread or search) after counting messages
• Spotify auto-play via Web API with device detection and graceful fallback to browser search
• Command history and real-time output UI
• Secure secrets handling via .env and .gitignore

Project Structure

• Backend: Vyn/backend (FastAPI, Mongo, Gmail API, Spotify API)
• Frontend: Vyn/frontend (Vite, React 19, TS, Tailwind v4)

Quick Start

1. Backend

• Create Vyn/backend/.env from the example:
  • Copy Vyn/backend/.env.example to Vyn/backend/.env
  • Fill values for Mongo and any optional integrations (OpenAI/Gmail/Spotify)
• Install deps and run:
  • Windows PowerShell: create a venv and pip install -r Vyn/backend/requirements.txt
  • Start server: uvicorn main:app --host 0.0.0.0 --port 8000 (from Vyn/backend)

2. Frontend

• From Vyn/frontend, install and run:
  • npm install
  • npm run dev
• The dev server runs at http://localhost:3000 and proxies /api to http://localhost:8000

Environment Variables (Backend)

See Vyn/backend/.env.example for all keys. Do not commit the real .env.

• MongoDB
  • MONGO_URI
  • DB_NAME
• OpenAI (optional)
  • OPENAI_API_KEY
• Gmail (optional)
  • GMAIL_TOKEN_PATH (defaults to token.json)
  • Place Google OAuth credentials.json and run python scripts/gmail_auth.py to generate token.json
• Spotify (optional)
  • SPOTIFY_CLIENT_ID / SPOTIFY_CLIENT_SECRET / SPOTIFY_REDIRECT_URI
  • Run python scripts/spotify_auth.py to cache a token to spotify_token.json

Security and Secrets

• This repo is configured to ignore env files and token caches:
  • Vyn/backend/.env
  • Vyn/backend/credentials.json
  • Vyn/backend/token.json
  • Vyn/backend/spotify_token.json
  • Vyn/frontend/.env*
• If a secret was ever committed in history, rotate it immediately and force-remove from git history if publishing.

Development Notes

• Backend main app: Vyn/backend/main.py
• Primary endpoint: POST /api/command orchestrated by the CommandRouter
• Voice mapping endpoint: POST /api/ai/map (optional OpenAI fallback)
• Frontend voice assistant listens globally for the wake word and debounces execution

Troubleshooting

• CORS/proxy: Frontend proxies /api to http://localhost:8000 via vite.config.ts
• Spotify requires an active device and Premium for playback via Web API; otherwise falls back to opening search
• Gmail OAuth: ensure redirect URIs match and test users are whitelisted when the app is unverified

License

See Vyn/LICENSE.