robot-perception

Solid

Comprehensive best practices for robot perception systems covering cameras, LiDARs, depth sensors, IMUs, and multi-sensor setups. Use this skill when working with RGB image processing, depth maps, point clouds, sensor calibration (intrinsic, extrinsic, hand-eye), object detection, semantic segmentation, 3D reconstruction, visual servoing, or perception pipeline optimization. Trigger whenever the user mentions OpenCV, Open3D, PCL, RealSense, ZED, OAK-D, camera calibration, AprilTags, ArUco markers, stereo vision, RGBD, point cloud filtering, ICP registration, coordinate transforms, camera intrinsics, distortion correction, image undistortion, sensor streaming, frame synchronization, or any computer vision task in a robotics context. Also covers multi-camera rigs, time synchronization across sensors, perception latency budgets, and production deployment of perception pipelines.

AI & Automation 251 stars 37 forks Updated 6 days ago Apache-2.0

Install

View on GitHub

Quality Score: 89/100

Stars 20%

Recency 20%

100

Frontmatter 20%

Documentation 15%

100

Issue Health 10%

License 10%

100

Description 5%

100

Skill Content

# Robot Perception Skill ## When to Use This Skill - Setting up and configuring camera, LiDAR, or depth sensors - Building RGB, depth, or point cloud processing pipelines - Calibrating cameras (intrinsic, extrinsic, hand-eye) - Implementing object detection, segmentation, or tracking for robots - Fusing data from multiple sensor modalities - Streaming sensor data with proper threading and buffering - Synchronizing multi-sensor rigs - Deploying perception models on robot hardware (GPU, edge) - Debugging perception failures (latency, dropped frames, misalignment) ## Sensor Landscape ### Sensor Types and Characteristics ``` Sensor Type Output Range Rate Best For ───────────────────────────────────────────────────────────────────────── RGB Camera (H,W,3) uint8 ∞ 30-120Hz Object detection, tracking, visual servoing Stereo Camera (H,W,3)+(H,W,3) 0.3-20m 30-90Hz Dense depth from passive stereo Structured Light (H,W) float + RGB 0.2-10m 30Hz Indoor manipulation, short range ToF Depth (H,W) float + RGB 0.1-10m 30Hz Indoor, medium range LiDAR (spinning) (N,3) or (N,4) 0.5-200m 10-20Hz Outdoor navigation, mapping LiDAR (solid-st.) (N,3) 0.5-200m 10-30Hz Automotive, outdoor IMU (6,) or (9,) N/A 200-1kHz Orientation, motion estimation Force/Torque (6,) float N/A 1kHz+ Contact detection, force control Tacti...