Mira

Hands-free web navigation from a plain webcam: head-pointing plus facial gestures, processed entirely on-device.

Mira lets someone drive a web page without a mouse, keyboard, or touch. Your head aims a cursor, an eyebrow raise clicks and drags, and an on-screen keyboard lets you type, all from the laptop camera you already have, with no video ever leaving your machine. It exists for people with motor impairments who can't use a conventional pointer, and for anyone who wants to interact with a page hands-free.

Status: early but working. The head-pointing engine, gesture clicks, edge-scrolling and on-screen keyboard all run today in the playground. The monorepo (@mira/core, @mira/sdk, @mira/extension, @mira/playground) is the scaffold for turning that prototype into a drop-in library and a browser extension. See ROADMAP.md for what's done and what's next, and ARCHITECTURE.md for how it's built.

What works today

• Head-pointing cursor. A rock-steady pointer driven by the MediaPipe FaceLandmarker head-pose matrix, not by chasing a jittery gaze estimate.
• DOM-snap targeting. The cursor snaps to the nearest interactive element (buttons, links, fields) with hysteresis and switch-debounce, so a noisy signal still selects reliably among discrete targets instead of needing pixel accuracy.
• One unified gesture. Raise an eyebrow to press, hold to drag (sliders), relax to click.
• Edge-zone scrolling. Tilt your head toward a screen edge to scroll, dwell-gated and ramped so it doesn't run away.
• On-screen keyboard. Head-point and eyebrow-click the keys; it pops automatically when a text field is focused.
• Signal conditioning. Median-of-5 (which kills blink and garbage frames) into a One Euro filter (Casiez 2012), tuned so webcam jitter doesn't masquerade as motion.

Privacy

All face processing runs on-device: the MediaPipe model is served from the same origin and inference happens in the browser. No camera frames, landmarks, or video are transmitted anywhere. The one deliberate exception, never on by default, is optional speech-to-text dictation, which (via the browser's Web Speech API) does send audio off-device; it is opt-in, clearly disclosed, and complements rather than replaces the on-screen keyboard. See the note in ROADMAP.md.

How it's meant to ship

The same core engine serves two audiences:

• A drop-in library (npm plus a CDN UMD bundle) that a site can include so its own users can enable hands-free control. UI renders inside a Shadow DOM so host CSS can't break it and it can't leak into the host.
• A browser extension (Manifest V3, Chrome and Firefox) that injects the engine into any page. This is the primary vehicle for the accessibility goal, since a site-included script can only help on sites that chose to include it.

Run the playground

The webcam API (getUserMedia) only works in a secure context, meaning HTTPS or exactly http://localhost, so the playground is served over HTTPS with a local self-signed cert. The page loads the engine from @mira/core's built output, so build it first.

# 1. install deps and build the engine (the playground imports @mira/core's dist)
pnpm install
pnpm -F @mira/core build

# 2. one-time: generate the local TLS cert (gitignored, never committed)
cd apps/playground
openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 365 \
  -subj "/CN=localhost" \
  -addext "subjectAltName=DNS:localhost,IP:127.0.0.1"

# 3. serve over HTTPS (serves the repo root, from any directory)
python3 serve.py

Then open https://localhost:8443/apps/playground/mira.html in Chrome. Because the cert is self-signed you'll see a one-time "Your connection is not private" warning; choose Advanced then Proceed, then grant the camera prompt. This is expected on your own machine.

License

Apache-2.0.