Closed-Loop Simulator Adapters

RoboCI closed-loop adapters connect external simulators to the existing runner and artifact pipeline:

simulator -> Observation -> agent/controller -> Action -> simulator.step() -> Observation

The runner still writes the same core artifacts used by reports, replay, API routes, and the dashboard:

  • episodes/<scenario_id>.jsonl
  • episodes/<scenario_id>.steps.jsonl
  • metrics.json
  • summary.json
  • failures.json
  • replay/<scenario_id>.*

Each step frame remains compatible with the existing episode schema (t, ego, actors, action, events). External adapters can also include sensors, calibration, robot, sim_state, and info.

Adapter Interface

Simulator adapters live under roboci/simulators/ and use shared primitives from roboci.simulators.base:

  • Observation: timestamp, cameras, LiDAR, ego or robot state, calibration, optional IMU/GPS, simulator metadata, events, and action.
  • Action: driving controls, target speed or trajectory, drone velocity, manipulator joints and gripper, and mobile robot twist fields.
  • SimState: pose, velocity, actor states, collision/contact events, route or task progress, and diagnostics.

roboci/simulators/* is the source of truth for adapter implementations. roboci/adapters/simulators/* exists only as a backward-compatible import layer for earlier extension code; those files re-export the real adapters and should stay logic-free.

Adapters may return the legacy dict observation shape or the newer tuple shape:

sim.connect(config)
obs = sim.reset(scenario)
while not done:
    action = agent.act(obs)
    obs, sim_state, done, info = sim.step(action)

connect(config), reset(scenario), step(action), is_done(), and close() are abstract SimulatorAdapter methods. The runner normalizes both reset/step result forms into RoboCI artifact frames.

Registry

The simulator factory accepts either a string or a config mapping:

create_simulator("mock")
create_simulator({"name": "carla", "host": "localhost", "port": 2000})

Registered names:

  • mock
  • replay_sim alias for the deterministic mock/replay-compatible simulator
  • ros2
  • carla
  • isaac
  • gazebo
  • Python simulator paths such as examples/simulators/simple_simulator.py
  • Native simulator sources or executables

Scenario Config

Existing scenarios can keep the string form:

simulator: mock

External simulator scenarios can use a mapping plus sensors and initial_state:

simulator:
  name: carla
  host: localhost
  port: 2000
  sync: true
  fixed_delta_seconds: 0.05

sensors:
  cameras:
    - name: front
      type: rgb
      width: 1280
      height: 720
      fov: 90
      transform:
        x: 1.5
        y: 0.0
        z: 1.6
        roll: 0
        pitch: 0
        yaw: 0
  lidar:
    name: roof_lidar
    channels: 64
    range_m: 80
    points_per_second: 100000

initial_state:
  pose:
    x: 0
    y: 0
    z: 0
    yaw: 0
  speed: 0

ego.start is still supported and remains the common path for existing marketplace scenarios. If a scenario has initial_state.pose but no ego, RoboCI derives ego.start so metrics and replay stay compatible.

Mock

The mock simulator remains deterministic and SDK-free, but it is domain-aware:

  • Driving and ground-robot scenarios keep the legacy planar x, y, yaw, and speed kinematics.
  • Drone scenarios infer task_type: drone from drone marketplace/domain metadata and expose z, altitude, vx, vy, and vz in ego, plus a robot.pose and robot.velocity payload. Drone actions can command vx, vy, vz, yaw_rate, linear_velocity, or target_speed.
  • Manipulator scenarios infer task_type: manipulator from manipulator marketplace/domain metadata and expose robot.joint_positions, robot.joint_velocities, robot.gripper, and robot.end_effector_pose. Manipulator actions can command joint_positions, joint_velocities, and gripper.

The mock simulator is still a smoke-test surface, not a replacement for Isaac Sim, Gazebo, CARLA, or hardware-in-the-loop validation.

CARLA

The CARLA adapter:

  • Connects with host, port, and timeout_seconds.
  • Loads map or world when specified.
  • Applies synchronous mode and fixed_delta_seconds when sync: true.
  • Spawns the ego vehicle from spawn_point_index when provided, otherwise from initial_state.pose or ego.start.
  • Attaches configured cameras and LiDAR.
  • Converts driving Action fields into carla.VehicleControl.
  • Collects collision, lane invasion, offroad, red-light violation, actor state, and route progress data when available. Red-light violations require the ego vehicle to still be moving while affected by a red signal, so a vehicle stopped at a red light is not penalized.
  • Reports event-sensor attach/listen failures through runtime warnings and adapter_warnings diagnostics. Set strict_event_sensors: true to fail reset when collision or lane-invasion event sensors cannot attach.

CARLA is imported lazily. Unit tests do not require CARLA. Live integration requires the CARLA Python API on PYTHONPATH.

simulator:
  name: carla
  host: localhost
  port: 2000
  map: Town03
  sync: true
  fixed_delta_seconds: 0.05
  vehicle_blueprint: vehicle.tesla.model3

Turnkey Docker Compose smoke path:

docker compose --profile carla up --build carla carla-roboci

The carla service runs carlasim/carla:0.9.15 with offscreen rendering. The carla-roboci service builds the carla-runner Docker target, checks for the CARLA Python API, waits for carla:2000, and runs:

roboci run --suite examples/suites/carla.yaml --agent examples/agents/rule_based_agent.py --sim examples/simulators/carla_town03.yaml

Inside Compose, ROBOCI_CARLA_HOST=carla and ROBOCI_CARLA_PORT=2000 override the manifest's local host/port values. On a local CARLA install outside Compose, keep the manifest defaults or set those environment variables to your host and port.

The default runner image does not bundle the CARLA Python API because published CARLA clients are platform- and Python-version-specific. For strict live CARLA validation, use a prepared Linux/GPU runner and set ROBOCI_CARLA_PYTHON_PACKAGE to the matching CARLA Python API wheel or package for that runner; without it, carla-roboci exits before the scenario run with a setup message.

Isaac Sim

The Isaac adapter is dependency-light and mockable. It supports task_type values:

  • driving
  • drone
  • manipulator
  • mobile_robot

It can load a stage with stage_usd when running inside Isaac Sim, or use injected world, controller, and reader hooks in tests.

simulator:
  name: isaac
  task_type: manipulator
  stage_usd: /path/to/workcell.usd

sensors:
  cameras:
    - name: wrist
      type: rgb
  lidar:
    name: depth
    type: depth

Action mapping:

  • Driving: steering, throttle, brake, gear, handbrake, target speed, trajectory.
  • Drone: vx, vy, vz, yaw_rate.
  • Manipulator: joint_positions, joint_velocities, gripper.
  • Mobile robot: linear_velocity, angular_velocity.

Isaac Sim modules are imported lazily. Live integration should run in Isaac Sim's Python environment.

Gazebo

The Gazebo adapter uses ROS 2 topics and optional services:

simulator:
  name: gazebo
  task_type: mobile_robot
  camera_topics:
    - /camera/image_raw
  lidar_topics:
    - /points
  odom_topic: /odom
  cmd_vel_topic: /cmd_vel
  reset_service: /reset_simulation
  step_service: /step_simulation

By default, RoboCI publishes mobile robot actions as geometry_msgs/Twist on cmd_vel_topic. Manipulator and drone command hooks are available through joint_command_topic, joint_command_builder, and drone_velocity_topic.

Set event_topics to subscribe to std_msgs/String event feeds. Each message can be a JSON object or list of objects, for example {"type":"red_light_violation","traffic_light_id":"main"}.

Gazebo reset and step spins wait for odometry plus every configured camera and LiDAR topic before returning, up to timeout_seconds. If no camera or LiDAR topics are configured, odometry is the required readiness signal.

ROS 2/Gazebo modules are imported lazily. Live integration requires a sourced ROS 2 environment with rclpy, geometry_msgs, sensor_msgs, nav_msgs, std_srvs, and std_msgs.

Smoke Tests Vs Integration Tests

The default test suite uses mocks and must pass without CARLA, Isaac Sim, ROS, or Gazebo installed:

python -m pytest tests/test_simulator_adapters.py

Live simulator tests should be gated explicitly:

ROBOCI_RUN_CARLA_TESTS=1 python -m pytest tests/test_simulator_adapters.py
ROBOCI_RUN_ISAAC_TESTS=1 python -m pytest tests/test_simulator_adapters.py
ROBOCI_RUN_GAZEBO_TESTS=1 python -m pytest tests/test_simulator_adapters.py

Only set those variables in environments where the relevant simulator stack is installed and reachable. The .github/workflows/simulator-integration.yml workflow is manual and targets self-hosted simulator runners so real CARLA, Isaac Sim, and Gazebo checks can run outside the ordinary dependency-light CI job.

Install Extras

RoboCI publishes simulator-named extras so environment manifests can declare intent:

pip install -e ".[carla]"
pip install -e ".[isaac]"
pip install -e ".[gazebo]"

These extras are intentionally dependency-light in pyproject.toml. CARLA, Isaac Sim, ROS 2, and Gazebo are distributed through simulator-specific installers and runtime environments rather than ordinary PyPI packages. Install the simulator SDK separately, then use the matching extra in project or CI metadata to make the adapter requirement explicit.

Adding a Simulator Adapter

When contributing a new built-in adapter:

  1. Add the adapter under roboci/simulators/.
  2. Register the simulator name in roboci/simulators/registry.py.
  3. Accept either string configuration or a mapping with name.
  4. Import optional simulator SDKs lazily inside the adapter so python -m pytest works without the SDK installed.
  5. Normalize observations into the shared Observation, Action, and SimState fields from roboci.simulators.base.
  6. Preserve core episode compatibility by writing t, ego or robot, actors, action, and events in every step.
  7. Add mockable unit tests under tests/test_simulator_adapters.py.
  8. Add a scenario example under examples/scenarios/ and a simulator manifest under examples/simulators/ when it should appear in launch options.
  9. Document config keys, required external services, and live-test environment variables in this guide.

Artifact Compatibility

The runner adds simulator metadata to each scenario entry in metrics.json:

{
  "simulator": {
    "name": "carla",
    "task_type": "driving",
    "sensor_config": {},
    "config": {"name": "carla", "host": "localhost", "port": 2000}
  },
  "task_type": "driving",
  "sensor_config": {}
}

Step frames may include:

  • sensors.cameras
  • sensors.lidar
  • calibration
  • robot
  • sim_state
  • info

Failure detection and metrics continue to use events, ego, actors, and action, so reports and dashboard views remain backward compatible.