All projectsPersonal project
Kinetic
Control a robot arm with your hand
A browser app where a simulated robot arm copies your hand movements through your webcam.
- Role
- Solo project
- Category
- Computer Vision
- Status
- Prototype
- Stack
- React, TypeScript, Three.js, MediaPipe, Rapier, Vite
(01) What it is
The project
Your webcam tracks your hand and a simulated robot arm copies it: position, palm direction, wrist twist and every finger. Pinch or make a fist to pick things up on a physics simulated table.
You can record what you do, replay it and export it as JSON, which is the kind of data people use to train robots. It all runs locally in the browser with no backend.
(02) Details
How it works
- 01Closed form inverse kinematics for the arm and fingers
- 02MediaPipe hand tracking in a Web Worker, smoothed with a One Euro filter
- 03Rapier physics so grabbing objects actually works
- 04Recordings export as structured JSON
(03) Screenshots
