RoboCI Documentation

This directory explains how RoboCI works from the inside out. Start with the architecture overview, then use the area-specific guides when changing code.

Documentation Map

  • Architecture: system model, major components, control flow, and artifact flow.
  • Codebase Guide: package-by-package explanation of the Python and frontend code.
  • CLI Guide: commands, exit codes, and common workflows.
  • API Guide: FastAPI endpoints and the local file-backed API model.
  • Data Formats: scenario, suite, run artifact, replay, failure, and open-loop formats.
  • Scenario Marketplace: installing, editing, and creating marketplace scenario packs.
  • ROS2 Bridge: JSON topic bridge for external ROS2 robot and simulator loops.
  • Closed-Loop Simulator Adapters: CARLA, Isaac Sim, Gazebo, and adapter contracts.
  • Simulator Support Matrix: v0.1 support status for mock, Python, CARLA, Isaac, Gazebo/ROS2, and hardware-in-loop surfaces.
  • Frontend Dashboard: Next.js structure, pages, components, data fetching, and common UI pitfalls.
  • Regression System: baseline comparison, gates, regression.json, and dashboard behavior.
  • Metrics: proof-style metric formulas, examples, edge cases, limitations, and test references.
  • Launch Demos: public-alpha demo flows for collision regression, open-loop evaluation, GitHub Actions, and dashboard replay.
  • Demo Capture Guide: terminal, dashboard, replay, failure-tagging, and regression capture instructions.
  • Extending RoboCI: adding simulators, agents, metrics, open-loop tasks, and dashboard panels.
  • Testing: test layout, how to run tests, and what each test protects.
  • Operations: local development, serving, generated files, troubleshooting, and environment variables.

High-Level Summary

RoboCI is a local-first evaluation framework for physical AI. It can run:

  • Closed-loop evaluations, where an agent interacts with a simulator step by step.
  • Open-loop evaluations, where a model predicts outputs for recorded dataset samples.
  • Regression comparisons, where current results are compared to a baseline run.
  • Replay and failure analysis, where each run produces local artifacts consumed by reports, the API, and the dashboard.

The project intentionally avoids a database for the MVP. The source of truth is the runs/ directory.