Embedded Bring-Up
Orin Nano / JetPack setup path, camera bring-up, Python environment, and hardware dependencies documented for repeatable workcell setup.
Robotics hardware/software integration
xArm Orin Vision-Gated Cube Grab
A Jetson Orin Nano workcell uses OpenCV confidence-gated cube detection to trigger a safe physical xArm primitive through the Hiwonder controller, with servo readback checks, trial logs, and validation-report tooling.
What this proves
Embedded robotics integration with test evidence.
This proof of concept connects perception, arm control, safety primitives, and validation into a small but real hardware-in-the-loop robotics system. It is intentionally practical: detect one cube reliably, gate motion on confidence and stability, command the arm, and write evidence about what happened.
- Compute
- Jetson Orin Nano embedded Linux environment
- Perception
- OpenCV color segmentation, ROI, brightness, confidence, and jitter gates
- Robot
- Hiwonder xArm 1S controlled through USB HID / serial command layers
- Evidence
- JSONL trial logs, servo readbacks, pose-safety tests, report generation
Demo video
Vision gate triggers the cube-grab primitive.
The demo shows the physical workcell running the vision-triggered sequence: OpenCV detects the target cube inside the work surface ROI, the gate passes, and the arm executes the taught primitive.
System layers
Built as a workcell, not just a motion demo.
OpenCV Detection Gate
HSV color ranges, ROI restriction, minimum brightness, detection rate, mean confidence, center jitter, and annotated snapshots.
Arm Command Layer
Hiwonder protocol packets, USB HID report handling, serial fallback, battery reads, servo position reads, and safe primitive execution.
Safety and Readback
Measured pose ranges, missing-readback checks, max-delta limits, error tolerances, and per-step command/readback records.
Validation Tooling
JSONL trial logs, success/failure fields, latency, retries, failure modes, perception metrics, and HTML report generation.
Test Suite
Unit tests cover protocol packets, HID report parsing, pose safety, action primitives, color-detection ROI logic, and report rendering.
Next iteration
A clean base for deeper robotics work.
- Add the exact OpenCV detection screenshot as a project image asset.
- Run 25 supervised physical trials, then generate a real validation report.
- Add planar calibration from image pixels to workspace coordinates.
- Promote the control layer into a ROS 2 node once the primitive loop is stable.
- Add a lightweight visualizer for servo state, command timing, and readback deltas.