Operations¶

This guide covers local development, generated files, environment variables, and common troubleshooting.

Local Setup¶

Run setup commands from a cloned RoboCI repository root when using explicit examples/... paths. The packaged roboci demo, roboci demo --openloop, and roboci benchmarks ... commands resolve their bundled assets automatically after installation.

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -c constraints.txt -e ".[dev]"

Frontend local-development dependencies:

cd frontend
npm install

Local Node Runtime¶

During development in this workspace, a local Node environment may exist at:

.nodeenv/

It is ignored by git. If your machine already has Node and npm, you do not need .nodeenv.

Native Example Prerequisites¶

Native C++ and Rust examples are optional. C++ .cpp examples need g++ or clang++ on PATH; Rust .rs examples need rustc on PATH. ROS2 examples need a local ROS2 installation and the bridge setup in docs/ros2.md.

Dependency Constraints¶

Runtime and development dependencies use lower and upper bounds in pyproject.toml. CI and release smoke installs also use constraints.txt:

pip install -c constraints.txt -e ".[dev]"

When bumping dependencies, update both files together and run the full test, dashboard build, lint, closed-loop smoke, and open-loop smoke commands.

Docker And Devcontainer¶

The default Docker image is CLI-focused and does not include CARLA, Isaac Sim, Gazebo, ROS2, or other simulator SDKs. The dashboard profile builds a separate prebuilt Next.js image; it does not run npm install when the dashboard container starts.

Build and run local checks:

docker compose build roboci
docker compose run --rm roboci roboci doctor
docker compose run --rm roboci roboci demo
docker compose run --rm roboci roboci benchmarks run smoke

Start the optional dashboard profile:

docker compose --profile dashboard up --build api dashboard

The dashboard profile uses the prebuilt dashboard image and the Python image for the API. It still reads local file-backed artifacts from the mounted workspace. Browser requests use NEXT_PUBLIC_ROBOCI_API=http://127.0.0.1:8000; server-rendered dashboard requests inside the Compose network use ROBOCI_SERVER_API=http://api:8000.

Start the CARLA smoke profile:

docker compose --profile carla up --build carla carla-roboci

This builds a RoboCI runner image, checks for the CARLA Python API, waits for carla:2000, and runs examples/suites/carla.yaml through examples/simulators/carla_town03.yaml. It requires a Docker host capable of running carlasim/carla:0.9.15, typically with GPU support. The default runner image does not bundle the CARLA Python API because published CARLA clients are platform- and Python-version-specific; set ROBOCI_CARLA_PYTHON_PACKAGE to a matching wheel/package on prepared CARLA runners.

Doctor¶

roboci doctor

Use roboci doctor to check Python, RoboCI installation metadata, examples, runs/, Node/frontend dependencies, and API port availability. It exits non-zero only for required CLI setup failures: Python, RoboCI installation metadata, and examples. Missing optional frontend setup, a missing runs/ directory before the first run, a busy port, or sandbox-blocked port probes are advisory status lines.

Serve Backend¶

roboci serve

Default:

http://127.0.0.1:8000

Serve Frontend¶

cd frontend
npm run dev

Default:

http://127.0.0.1:3000/dashboard

Environment Variables¶

ROBOCI_RUNS_DIR¶

Force the API and run store to use a specific runs directory.

ROBOCI_RUNS_DIR=/absolute/path/to/RoboCI/runs roboci serve

ROBOCI_RUNS_DIR can point at a mounted network share when multiple trusted workstations need to inspect the same artifacts.

ROBOCI_API_TOKEN¶

Require a shared API token on routed API calls.

ROBOCI_API_TOKEN="$(openssl rand -hex 24)" roboci serve --host 0.0.0.0

Clients then send either Authorization: Bearer <token> or X-RoboCI-Token: <token>.

Dashboard fetches use NEXT_PUBLIC_ROBOCI_API_TOKEN. roboci dev sets it from ROBOCI_API_TOKEN automatically when both servers are launched together:

ROBOCI_API_TOKEN="$(openssl rand -hex 24)" roboci dev

For users without local Node, run the same development entry point through Docker Compose:

roboci dev --docker

ROBOCI_EXAMPLES_DIR¶

Force the API scenario catalog to use a specific examples directory.

ROBOCI_EXAMPLES_DIR=/absolute/path/to/RoboCI/examples/scenarios roboci serve

NEXT_PUBLIC_ROBOCI_API¶

Tell the dashboard where the API is.

NEXT_PUBLIC_ROBOCI_API=http://127.0.0.1:8000 npm run dev

For standalone frontend runs on a non-default port, pass Next.js' port flag after --:

NEXT_PUBLIC_ROBOCI_API=http://127.0.0.1:8000 npm run dev -- --port 3010

When using roboci dev, set the dashboard port with --frontend-port, NEXT_PUBLIC_ROBOCI_PORT, or ROBOCI_FRONTEND_PORT:

NEXT_PUBLIC_ROBOCI_PORT=3010 roboci dev

Set the same variable for production-style local starts:

NEXT_PUBLIC_ROBOCI_API=http://127.0.0.1:8000 npm exec next start
NEXT_PUBLIC_ROBOCI_API=http://127.0.0.1:8000 npm exec next start -- --port 3010

The Settings page is server-rendered dynamically and reports the browser-facing and server-side runtime API base URLs. If Settings shows different URLs than expected, restart the frontend process with NEXT_PUBLIC_ROBOCI_API and, when needed, ROBOCI_SERVER_API set in that process environment.

ROBOCI_SERVER_API¶

Set a server-side API URL for Next.js server-rendered dashboard fetches when it differs from the browser-facing URL.

ROBOCI_SERVER_API=http://api:8000 NEXT_PUBLIC_ROBOCI_API=http://127.0.0.1:8000 npm run dev

This is mainly useful in Docker Compose, where the browser reaches the API through the host-published port but the dashboard container reaches it through the service name.

ROBOCI_CARLA_HOST And ROBOCI_CARLA_PORT¶

Override CARLA adapter host/port values from scenario YAML. The CARLA compose profile sets these to carla and 2000 so the same example scenario can run inside the Compose network.

Remote Artifact Copies¶

RoboCI remains file-backed, but artifacts can be copied to a mounted network share, S3, or GCS:

roboci storage sync runs/latest /mnt/team-share/roboci/latest
roboci storage sync runs/latest s3://my-bucket/roboci/latest
roboci storage sync runs/latest gs://my-bucket/roboci/latest

Cloud destinations use the local aws or gsutil CLI and inherit that tool's authentication and retry behavior.

Generated Files¶

Ignored by git:

• .venv/
• .nodeenv/
• __pycache__/
• .pytest_cache/
• .roboci/
• roboci.egg-info/
• runs/
• baselines/
• artifacts/
• screenshots/
• recordings/
• captures/
• frontend/.next/
• frontend/node_modules/
• frontend/out/
• frontend/.turbo/

Tracked:

• Source files.
• Examples.
• Tests.
• Documentation.
• frontend/package-lock.json.

Troubleshooting¶

{"detail":"Not Found"}¶

You opened the FastAPI root:

http://127.0.0.1:8000/

Use:

http://127.0.0.1:8000/health
http://127.0.0.1:8000/docs
http://127.0.0.1:8000/runs

Dashboard Shows No Runs¶

Check the API:

http://127.0.0.1:8000/runs

If it returns [], set:

ROBOCI_RUNS_DIR=/absolute/path/to/RoboCI/runs roboci serve

Dashboard Shows Zeros¶

The newest run may be open-loop. Open-loop runs do not have closed-loop metrics like success_rate and safety_score.

The dashboard uses run_type to display open-loop and closed-loop metrics separately.

Regression Diff Is Empty¶

Run:

roboci baseline create smoke --from runs/<baseline-run-id>
roboci compare --current runs/latest --baseline-name smoke

Normal roboci run writes a gate result with empty deltas. Only roboci compare computes baseline deltas.

If no compatible baseline exists yet, run the same workflow once more, create a named baseline from the earlier run, and compare the newer run against that baseline.

Frontend Cannot Fetch API¶

Check:

curl http://127.0.0.1:8000/health

Then start frontend with:

NEXT_PUBLIC_ROBOCI_API=http://127.0.0.1:8000 npm run dev

For next start, also pass NEXT_PUBLIC_ROBOCI_API to that start command. If the frontend runs inside a container, set ROBOCI_SERVER_API to the API address reachable from that container. Reusing a build with a different runtime API without setting the variable on the running process can make server-rendered pages and client-side actions disagree about the API origin.

Next Build Fails In A Restricted Sandbox¶

RoboCI's dashboard build script uses Webpack (next build --webpack) so production-style dynamic routes, including /failures/[failure_id], work under next start. If a restricted sandbox still blocks worker process operations, run the build outside that restricted environment.

CI¶

Example GitHub Actions workflow:

.github/workflows/roboci.yml

It installs RoboCI, runs roboci benchmarks run smoke, regenerates the markdown report, uploads roboci-smoke-run.zip, and posts the markdown summary back to the pull request. .github/workflows/roboci-example.yml remains a fuller repository-maintainer workflow that also runs pytest, frontend lint/build, and demo checks.