Spectra
Real time object detection and tracking
A computer vision platform that detects and tracks objects in video and fires events when they cross zones.
- Role
- Solo project
- Category
- Computer Vision
- Status
- Complete
- Stack
- Python, PyTorch, OpenCV, FastAPI, WebSockets, Next.js, Three.js
(01) What it is
The project
Spectra takes a video feed, runs YOLO detection and segmentation on the GPU, tracks each object across frames, and turns that into events, like a person crossing a line. A FastAPI server streams the results over WebSockets to a Next.js app that draws the overlays on top of the video.
The biggest lesson was to measure instead of guess. Back to back benchmarks said one inference took 7.1 ms, but in the real pipeline it was closer to 24 ms because the GPU sits idle between frames. The supposedly lightweight SSDlite model also turned out to be the slowest one on my hardware. I also tracked down and fixed a GPU race condition that crashed the server when you switched models.
(02) Details
How it works
- 0124.4 ms p50 and 29.2 ms p99 from decoding a frame to the result reaching the browser
- 02My own ByteTrack style tracker with a Kalman filter, at 0.24 ms per frame for 10 objects
- 03Zone and tripwire rules use hysteresis and cooldowns, so one crossing fires one event
- 04Detection can also run fully in the browser on WebGPU, so camera frames never leave your device
- 0592 pytest tests, including access control, CSRF and SSRF protection
(03) Screenshots
