Two boards, one wire pair
OKO-1 splits the job across two ESP32 chips. The ESP32-CAM (AI-Thinker, OV2640) sits on the tilting head next to your webcam and only looks for faces. The main ESP32 sits in the housing, drives both servos and talks to your computer over USB for settings. They share one UART at 921 600 baud. Because the tracking camera rides on the head, the loop is closed: every correction changes what the camera sees next.


Face detection at 7 fps on a $6 board
The camera sketch uses Espressif’s esp-dl face detector from the Arduino core esp32 2.0.17. The headers it needs were removed in core 3.x, so the version matters. The detector costs about 17 µs per input pixel, which makes a full 320 × 240 frame slow. The firmware therefore has two paths:
- Search: the whole frame is scanned only when no face is locked, or right after a miss.
- Fast path: once a face is found, only a square window around it is cut out, averaged down to a small input (each output pixel is the mean of 4 samples, which also removes noise) and searched again. That is what keeps detection near 7.2 per second.
- The sensor runs free with two frame buffers and the newest frame wins, so the loop never waits for the camera.
- With PSRAM it works at QVGA (320 × 240); without PSRAM it still tracks at QQVGA (160 × 120). Wi-Fi is never started.
A text protocol you can read
Every processed frame produces exactly one line. No pictures leave the camera during tracking, only the face box and timing:
F <cx> <cy> <w> <score*100> <W> <H> <ms> face found N <W> <H> <ms> no face in this frame
cx cy w are the face centre and width in pixels, W H the frame size and ms how old the frame was when the line was sent. That last number is what makes the control law work.
The control law: correct from where the camera was
A naive tracker takes the error in the latest picture and moves by that amount. With a slow detector this overshoots: by the time the report arrives the head has already moved. OKO-1 records the servo positions every 10 ms for the last 640 ms, looks up where the head pointed when the frame was taken, and sets the target relative to that position. Nothing is integrated, so the loop stays calm at any frame rate.
// main ESP32, on every face report from the camera dpp = fov / max(W, H) // degrees per pixel, fov = 56° (OV2640 stock lens) ex = (cx - W/2) * dpp - aimX // horizontal error, degrees ey = (H/2 - cy) * dpp - aimY // vertical error, degrees back = (frameAge + lag) / 10 ms // lag = 80 ms by default pan0 = panHistory[back] // where the head pointed when the frame was taken if |ex| > dead: panTarget = pan0 + ex * kp // kp = 0.70, dead = 2° (same for tilt)
- Self-check: if corrections keep making the error bigger, the firmware flips that axis by itself and saves it. A servo mounted the other way round fixes itself in a few seconds.
- Aim offset: your webcam sits a few centimetres from the tracking camera.
aimx/aimyshift the target so the face is centred in the webcam, not in the ESP32-CAM. - Home: after 6 s without a face the head glides back to the middle.
- A predictive alpha-beta filter exists as an experiment (
set pred), but it is off after every start: in live tests the plain law felt better.
Smooth motion with a critically damped follower
The servos never jump to the target. They follow it like a critically damped spring: the move starts gently, slows down gently and never overshoots. About 90 % of a move is done after smooth ms (500 by default), capped at 90°/s. A new target in the middle of a move just bends the path. The PWM value is only rewritten when it changes, which removes the classic hobby-servo jitter. MG90S scale: about 10.3 µs per degree.
Powering two servos from a laptop port
- Two MG90S pulling current dip the USB 5 V rail. The firmware disables the ESP32 brownout reset, otherwise the board would reboot on every fast move.
- At power-up the servos start one after another, 350 ms apart, to lower the current peak.
- A 470-1000 µF capacitor across 5 V and GND helps a weak port.
- The camera runs from the 3V3 pin of the main board; its own 5 V pin stays empty.
Tuning commands
save.| Command | What it does |
|---|---|
status | Live state: LOCK / SEARCH / NOLINK, angles, fps |
track 0 / 1 | Tracking off / on |
center | Both servos to the middle (for fitting the horns) |
pan <deg> / tilt <deg> | Manual move |
flip pan | tilt | Reverse an axis |
set kp | dead | speed | smooth | aimx | aimy … | Tuning; `show` lists everything |
cam <rot | score | in | roi | perf …> | ESP32-CAM settings, passed through |
bridge | USB ↔ camera link for flashing the ESP32-CAM |
log | Last 256 camera reports with servo angles |
save | Store settings in flash |
Maker questions
Why does the ESP32-CAM face detection not compile on Arduino core 3.x?
The esp-dl face detector headers used by the classic CameraWebServer example were removed from the Arduino-ESP32 core in 3.x. OKO-1 pins core 2.0.17, where they are still included.
Can one ESP32-CAM drive the servos directly?
It can, but the camera board has few free pins and shares its power with the sensor. OKO-1 keeps the camera board only for detection and lets a second, cheap ESP32 do PWM, smoothing and USB settings. It is more robust and still costs about $4.50 extra.
How do you stop servo tracking from overshooting?
Correct from where the head pointed when the frame was taken, not from where it is now. OKO-1 stores the servo positions every 10 ms and uses the frame age from the camera to look back, then moves by 70 % of the error with a 2° dead zone and critically damped easing.
Does it work without PSRAM?
Yes. Without PSRAM the camera works at 160 × 120 and still tracks a face at desk distance.
Fork it, port it, improve it
MIT licence. Both sketches, the flashing scripts and a Python monitor with live telemetry are in the repository.