Touchless Instrument . Touchdesigner + ableton· 2026

TD Air MIDI

Hand-Gesture-to-Music System in TouchDesigner

Imagine being able to play notes without touching any physical instrument — just by moving your fingers in front of a camera, as if playing the keys of an invisible piano, with near-zero latency.

This project uses MediaPipe as the underlying hand-tracking infrastructure, but the core of the work is two custom-built base COMPs, designed and developed from scratch:

- Finger_Flex_Pulse — takes raw, normalized hand-tracking data and converts each finger's movement into a trigger signal (a 0/1 pulse for every tap, like pressing a piano key) and a flex signal (a numeric range representing how bent the finger is). Left and right hands are fully distinguished from one another, and detection error has been significantly reduced.

- FingerNotes — builds on the output of the previous system, converting each of the ten fingers' signals into a musical note within a range you define yourself, with a live visual of ten labeled piano keys shown as a background overlay.

The output of this chain is sent to Ableton Live via MIDI (using LoopMidi), while a live visual simultaneously displays each finger's position and tap moment on screen.

FingerNotes

Inside FingerNotes, each of the ten fingers (Finger L1–L5 for the left hand, Finger R1–R5 for the right) has its own dedicated Note parameter. Selecting it opens a full chromatic list spanning multiple octaves, so any pitch can be assigned to any finger. The COMP's built-in piano-key visual updates live to show exactly which note is currently mapped to each finger, making the full range easy to see and adjust at a glance.

Everything is designed for minimal processing and minimal delay, so that playing genuinely feels instantaneous.

This project is a foundational platform, not a closed final product — a starting point for building instruments without physical hardware, interactive music education, and responsive, exhibition-ready installations. If you're looking for a reliable, low-latency infrastructure for your own work in interactive art or the NIME space, this is a solid place to start.

Tools

TouchDesigner, Ableton Live, LoopMIDI

Tracking

MediaPipe Hands

Output

MIDI via LoopMidi

Year

2026