Demo Capture Guide

Use this guide to refresh real demo media for the README. Do not create mock screenshots, synthetic UI images, or edited success states. Capture the current app running against real RoboCI artifacts.

Asset Targets

The README expects these media files:

assets/dashboard.png
assets/replay.png
assets/regression.png
assets/roboci-demo.gif

The assets/ directory is reserved for these launch captures.

Setup

Install the development dependencies and verify the CLI before recording:

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -e ".[dev]"
roboci doctor

Start the dashboard through Docker Compose. This builds the prebuilt dashboard image and does not require Node on the host:

docker compose --profile dashboard up --build api dashboard

Alternatively, use the CLI wrapper:

roboci dev --docker

For local dashboard development, install Node dependencies once and run the dev server:

cd frontend
npm install
cd ..
roboci dev

Use the dashboard and API URLs printed by the launch command. If the default ports are free, the dashboard is usually available at:

http://127.0.0.1:3000/dashboard

Terminal Eval Run

Record a terminal session that shows a real evaluation command completing and writing artifacts.

Suggested passing run:

roboci run \
  --suite examples/suites/smoke.yaml \
  --agent examples/agents/rule_based_agent.py \
  --sim mock

Suggested failure run for replay and tagging captures:

roboci run \
  --suite examples/suites/failing_collision.yaml \
  --agent examples/agents/collision_test_agent.py \
  --sim mock

Capture requirements:

  • Show the command, run id, final status, and artifact path.
  • Keep the terminal width wide enough that paths and summaries are readable.
  • Use the real command output; do not trim the recording into a misleading state.
  • Export the final short demo animation to assets/roboci-demo.gif.

Dashboard

Capture the dashboard after at least one passing run and one failing run exist.

Suggested page:

http://127.0.0.1:3000/dashboard

Capture requirements:

  • Show run summary cards and recent run rows populated from real runs/ artifacts.
  • Avoid empty-state screenshots unless the README is explicitly documenting setup only.
  • Save the capture as assets/dashboard.png.

Replay Viewer

Capture a scenario replay from a real run that has replay artifacts.

Suggested flow:

  1. Run the failing collision suite from the terminal eval section.
  2. Open the run list.
  3. Select the failing run.
  4. Open one scenario replay page.

Suggested URL pattern:

http://127.0.0.1:3000/runs/<run_id>/scenarios/<scenario_id>

Capture requirements:

  • Show the replay viewer loaded with real timeline data and the replay scene with ego path, actors, and event markers.
  • Show the event timeline with at least one real event, such as a near miss or collision.
  • Include enough surrounding UI to make the run and scenario context clear.
  • Save the capture as assets/replay.png.

Failure Tagging

Capture failure tagging from a run that contains detected failures.

Suggested flow:

  1. Run the failing collision suite from the terminal eval section.
  2. Open the failing run detail page.
  3. Open the failure or triage area.
  4. Add or update a tag, lifecycle status, owner, or fixed/regressed run reference using the dashboard UI.

Capture requirements:

  • Show a real detected failure and its tag, lifecycle status, owner, or triage state.
  • Confirm the edit is persisted by refreshing or revisiting the failure before recording the final take.
  • This workflow is part of the demo recording even though the README does not currently reserve a separate failure-tagging image.

Regression Explorer

Capture the regression view after comparing a current run with a baseline.

Suggested flow:

roboci baseline create smoke --from runs/<known-good-run-id>

roboci compare \
  --current runs/latest \
  --baseline-name smoke \
  --max-regression-percent 5

Then open the regression dashboard page or the compared run detail page.

Capture requirements:

  • Show real metric deltas from regression.json.
  • Include the closed-loop or open-loop section label so the evaluation mode is clear.
  • Include at least one visible diff card with previous value, current value, delta or status transition, severity, and affected scenario/sample when available.
  • If a scenario regressed from present to missing, show the success -> missing transition rather than a replay page with missing artifacts.
  • Save the capture as assets/regression.png.

Final Review

Before committing captured assets:

  • Re-run the commands shown in the recording on a clean checkout or fresh run directory.
  • Confirm every image matches the current UI.
  • Confirm the GIF is short, readable, and does not skip the result summary.
  • Do not commit generated runs/ directories or copied baselines/ data.