Webcam Theremin
Wave your hands in front of your camera and play a synthesizer in the air, like a theremin. Right hand sets the pitch, left hand sets the volume. The hand tracking and the sound both run entirely on your machine.
📷 100% private — your camera feed never leaves your deviceHow it works
A theremin is the instrument you play without touching it: you move your hands near two antennas and the pitch and volume follow. This version swaps the antennas for your webcam. It loads Google's MediaPipe hand-landmark model into your browser and runs it on your own GPU, finding twenty-one points on each hand about thirty times a second. The vertical position of your hand becomes a note, a second hand rides the volume, and a small synthesizer built with the Web Audio API turns that into sound. Every part of this happens locally, so there is no upload, no latency from a round trip, and nothing is recorded.
How does it run without a server?
The model is a few megabytes of weights that download once and then sit in your browser cache. Inference runs through WebAssembly and WebGL on your graphics card, which is why it keeps up with live video. The audio is generated by oscillators and filters that the browser ships natively. Put together, your camera, the tracking, and the sound stay on one machine: yours.
Things to try
Switch the scale to minor pentatonic or blues and you almost cannot hit a wrong note, which makes it fun to improvise. Pick a sawtooth wave and add reverb for a spacey lead. Pinch your thumb and finger together to add vibrato, the wobble a real theremin player makes with tiny hand movements. Switch pitch mode to continuous for the genuine sliding theremin sound, with no notes to snap to.
Is my camera uploaded anywhere?
No. The hand tracking and the synthesizer both run locally in your browser. Your camera feed never leaves your device and nothing is recorded or sent to a server.
How is the hand tracking done?
It uses Google's MediaPipe hand-landmark model running on your GPU through WebAssembly. It detects 21 points on each hand many times a second and maps their position to pitch and volume.
Do I need a webcam?
For the full experience, yes. But there is a mouse mode so you can play the same synth with your pointer if you have no camera or prefer not to grant access.
Why is there a short wait the first time?
The hand-tracking model is about 8 MB and downloads once. After that your browser caches it and the tool starts quickly.
It is silent or laggy. What helps?
Use a desktop browser with a real GPU, good lighting, and keep your whole hand in the frame. Chrome and Edge tend to give the smoothest tracking.
I build tools like this every day.
Senior full-stack engineer, available for senior or contract work, fully remote. See the rest of the lab or get in touch.