Dex — Holographic Spotify Assistant

Dex is a holographic music assistant built to be cast via projector to control Spotify through natural language voice and text commands. It runs locally on your machine with the intent of being casted via projector, a local AI brain via Ollama, and hand gesture control via webcam.

---

Features

• Natural language music control — voice or text commands interpreted by a local LLM
• Full Spotify integration — play albums, tracks, artists, liked songs, playlists, and mood-based searches
• Holographic UI — dark, futuristic interface with ambient album art colour, waveform visualiser, and HUD overlays
• Hand gesture control — pinch to click cursor via webcam using MediaPipe
• High quality TTS — Microsoft Neural voice (en-GB-RyanNeural) via edge-tts, played at native quality through ffplay
• Session memory — Dex remembers context across commands within a session
• 100% local AI — no OpenAI, no paid APIs. Runs on Ollama + llama3.2

---

Commands

You say	Dex does
"Pull up Views by Drake"	Plays the album
"Play Blinding Lights"	Finds and plays the track
"Throw on some Kendrick"	Plays Kendrick Lamar's catalogue
"Play my liked songs"	Shuffles your Spotify library
"Play my workout playlist"	Finds and plays that playlist
"Play something chill"	Finds a matching mood playlist
"What's playing?"	Tells you the current track
"Pause / Skip / Go back"	Playback controls

---

Stack

Layer	Technology
AI Brain	Ollama + llama3.2 (local)
Music	Spotify Web API via Spotipy
Voice	SpeechRecognition + Google STT
TTS	edge-tts (Microsoft Neural, free)
Audio playback	ffplay (zero resampling)
Gesture	MediaPipe + OpenCV
Server	Flask
UI	Vanilla HTML/CSS/JS

---

Requirements

• Python 3.10+
• Ollama running locally with llama3.2 pulled
• Spotify Premium account
• Spotify Developer app credentials
• ffmpeg installed (brew install ffmpeg)

---

Setup

1. Clone the repo

git clone https://github.com/jasecolino/DexMusic.git
cd DexMusic

2. Create your .env file

cp .env.example .env

Fill in your Spotify credentials in .env:

SPOTIFY_CLIENT_ID=your_client_id_here
SPOTIFY_CLIENT_SECRET=your_client_secret_here
SPOTIFY_REDIRECT_URI=http://127.0.0.1:5678/rest/oauth2-credential/callback
OLLAMA_MODEL=llama3.2
OLLAMA_BASE_URL=http://localhost:11434

3. Install dependencies

pip install -r requirements.txt
brew install ffmpeg

4. Pull the AI model

ollama pull llama3.2

5. Launch

bash dex_launch.sh

Dex will open in Chrome in fullscreen. Say "Hey Dex..." or press / to type a command.

---

Gesture Control (optional)

Run in a separate terminal window while Dex is running:

python dex_gesture.py

Show your hand to the webcam. Pinch your index finger and thumb together to move the cursor, release to click.

---

Spotify App Setup

1. Go to developer.spotify.com → Dashboard → Create App
2. Set redirect URI to: http://127.0.0.1:5678/rest/oauth2-credential/callback
3. Copy Client ID and Client Secret into your .env

---

Keyboard Shortcuts

Key	Action
Space	Play selected track
→	Skip
←	Previous
↑ ↓	Navigate tracklist
/	Open command bar
F	Toggle fullscreen

---

Project Structure

DexMusic/
├── DexV6.py          # Main Flask server, Spotify handlers, audio pipeline
├── dexBrain.py       # Ollama intent parser (AI brain)
├── dexIndex.html     # Holographic frontend UI
├── dex_gesture.py    # Hand tracking gesture controller
├── dex_launch.sh     # One-command launch script
├── requirements.txt  # Python dependencies
├── .env.example      # Environment variable template
└── .gitignore

---

Built with zero paid AI APIs. Runs entirely on your machine.