Sensor coverage

Estimate how well a set of sensors observes a fixed world — how much of the room and which objects are seen, and by how many sensors (0..N) — and search for a layout (how many, which type, placed where) that reaches a target coverage. Implemented in roqsim_sensors.coverage; the 2D heatmap needs the optional coverage extra (pip install 'roqsim_sensors[coverage]', already installed by make venv), the 3D render needs only MuJoCo. Rendering works headless without any setup — import roqsim selects an offscreen backend for the machine (see roqsim.gl.select_offscreen_gl()); set MUJOCO_GL only to override it.

There are two front doors onto the same core:

  • the plugin sensor_coverage_probe — a world-YAML toggle that reports the coverage of the sensors already in a world;

  • the CLI roqsim sensors coverage — a placement-search workbench that evaluates hypothetical candidate mounts and iterates toward a target.

Concepts

  • Field of view. Every sensor reduces to one SensorFov — a posed angular sector: a camera is a rectangular FRUSTUM, a lidar a CONE_BAND (azimuth × elevation band). It is extracted per sensor type by an adapter (see Architecture & porting playbook › Sensor coverage), so a camera’s FOV comes from its MJCF fovy/resolution and a lidar’s from its plugin defaults — never re-typed.

  • Coverage count. For each sample point, the number of sensors that see it, gated by range → angular FOV → line of sight (occlusion by walls/furniture, but not by the sensor’s own mount — see below). k=1 means “seen by ≥1 sensor”; k=2 is redundant coverage.

  • Sample targets. volume is a 3D grid of free interior points (room coverage); objects are points on object surfaces, labelled by geom name (are these objects seen?).

  • Orientation matters. With rpy = 0 a sensor points along +x (world), up = +z. To look down, a camera needs rpy: [0, 1.5708, 0]; an upright Livox (vertical band −7°..+52°) must be inverted (rpy: [3.14159, 0, 0]) or it sees almost nothing below it. Near-zero coverage is usually an orientation mistake.

  • Camera range is an assumption. A camera has no physical far range; the far you give it is a detection range and too generous a value inflates coverage (depth cameras default it to their clip_far).

The plugin (world-YAML toggle)

List sensor_coverage_probe in a world’s components: to compute coverage once (at configure) and write report.json plus a render; omit it for none. sensors: auto evaluates every MuJoCo camera in the world; give an explicit list for lidars/Livox or hypothetical placements.

components:
  - spawn_sensor: {model: mid360, name: cam_a, pos: [3, 1, 2.4], rpy: [3.14159, 0, 0]}
  - sensor_coverage_probe:
      sensors: auto              # or a list of {type, pos, rpy, config}
      target: {k: 1, frac: 0.95}
      sample: {volume: true, objects: true, resolution: 0.25, heights: [0.3, 1.0, 1.7]}
      out: coverage              # writes report.json + render(s) here
      render: both               # 3d | 2d | both | none
      palette: coverage          # 'coverage' (red 0->green many) | 'density' (light 0->dark many)

roqsim sim world.yaml --headless --steps 1

Adding a sensor the tool doesn’t know

If build_fov raises no adapter for '<type>', register one in roqsim_sensors/coverage/adapters.py (@register_adapter("<type>") returning a SensorFov) and add a CATALOG entry in catalog.py. The sensor plugin itself is never modified. See Architecture & porting playbook › Sensor coverage for the design.

A catalog entry states policy, not optics. Write cost, mount and description – how this device may realistically be deployed – and name the bundled model it describes. The optics are read from that model: fovy and resolution off the MJCF camera its manifest wires to the capture plugin, near/far off the manifest’s fov: block. So a camera’s field of view is stated once, in the model, and the same numbers drive spawn_sensor: {show_fov: true} and a coverage study. A lidar entry names no model and needs no optics at all – its adapter instantiates the plugin and reads the defaults it resolved.

Restating those numbers in the catalog is what an entry must not do – a copy cannot be kept true by attention, and a catalog that disagrees with the model it names is worse than one that says nothing – so there is deliberately no field to write one in. fov_overrides exists for the genuine exception – a study pinning a camera’s far, which is an analysis assumption rather than a property of the device.

Visualising a sensor’s FOV directly

Independently of coverage, spawn_sensor: {show_fov: true} draws a sensor’s field of view in the viewer/renders. Three paths, by what the model provides:

  • a camera mount (the RealSense/Zivid models) synthesises a translucent view frustum from the camera’s fovy/aspect spanning fov_near..``fov_range``, always clipped against world geometry into a visibility volume that stops at walls and objects (see below);

  • a camera-less lidar (Mid-360, Robin W1G) synthesises a translucent angular sector shell from the datasheet angles (h_min/h_max/v_min/v_max) in its manifest’s fov: block, using the very ray convention the capture plugin casts with – a full 360deg dome for the Mid-360, a bounded forward 120deg x 70deg wedge for the Robin W1G;

  • a camera-less model that ships a bundled _fov mesh reveals it, if neither of the above applies (none of the current models: the Zivid ships one but has a camera, so it takes the frustum path).

fov_near/fov_range default to the sensor model’s own fov: {near, far} block in its <model>.manifest.yaml (device knowledge lives with the device – Zivid 1.3..5 m, D435 0.28..6 m, Mid-360 0.1..40 m, Robin W1G 0.1..70 m), so show_fov: true alone draws the correct band; a world may override either per placement. fov_near sets the near cap of the drawn volume, so its shape is the valid detection band. A model that has a camera always synthesises its frustum from that camera, even the Zivid (which also ships a bundled _fov envelope) – the baked envelope stays hidden. The worlds/all_sensors_demo.yaml world shows every sensor’s FOV and runs a coverage probe.

A synthesised camera frustum is always clipped into a visibility volume that stops at walls and objects instead of passing through them (a fov_rays grid, default [32, 24], is cast from the camera against the world) – occlusion is unconditional, not a per-placement opt-in. It applies only to synthesised camera frustums; a bundled envelope and a lidar sector are drawn un-clipped. The clip is a static build-time snapshot of the world built so far, so list scene/floorplan plugins before the sensors; dynamic bodies occlude at their spawn pose and the volume does not update at runtime. This is a per-sensor visual (what one sensor can see); for the quantitative per-area overlap count across all sensors use the coverage probe with palette: density.

A sensor never occludes itself

A real device’s lens sits on the outside of its housing, but a MuJoCo <camera> sits at the pose the datasheet gives — millimetres behind the geom that models that face. A visibility ray therefore leaves the origin already inside the sensor’s own body, and without care the sensor’s own housing is the first thing it hits. So every FOV carries the body it is mounted on (SensorFov.body_exclude, from cam_bodyid/site_bodyid) and the ray cast excludes it — the same bodyexclude mechanism the lidar plugin’s exclude_body uses for a scanner’s own housing.

Worth stating because the symptom does not look like occlusion. The D435 mount’s d435_front sits 4.3 mm ahead of its camera, which unexcluded blocks the whole central cone while wide-angle fringe rays still escape — so a mounted camera reports a plausible but low number rather than an obvious zero, and a narrow long-range sensor (the Zivid, near 1.3 m) reports exactly 0 coverage in a room it sees perfectly well. A study that lets a mount occlude its own camera therefore under-reports every spawn_sensor-mounted sensor. A hypothetical placement (pos/rpy, not spawned) is unaffected: it has no body, because nothing of it exists to get in the way.