What it does
Speech recognition and fingertip tracking run separately; Jev chooses among bounded commands, shapes, colors, targets, and placements.
Playable demos & bots
jev-canvas
Speak an English or Ukrainian drawing command while pointing, and Jev translates it into an action on a tldraw canvas.
Bookmark: jev-canvas
Keep this in your collection.
Leave a noteWhat would you try with this?
: jev-canvas
Only you can see your notes.
Recording unavailable. Open the experiment ↗
How you can use it
You can borrow this approach to let people control an app with their voice and hands. A developer can adapt the code. It sends spoken words and a finger location to the TypeSafe AI service. TypeSafe then decides which shape or color to draw.
Your developer must provide an access key from OpenRouter. This key connects the software to the outside AI service. TypeSafe only decides what the spoken commands mean. The local app actually draws the shapes and tracks the hands.