Results Processing

Every RoboVAST execution produces a results directory with a well-defined layout. This page documents the output structure and how to postprocess and merge results using the vast results command group.

Output Structure

The results directory path is configured during vast init and stored in the .robovast_project file.

Top-Level Layout

<results-dir>/
└── <campaign-name>-<timestamp>/          # One per execution (e.g. dynamic_obstacle-2026-03-04-152130)
    ├── metadata.yaml                     # Campaign metadata (auto-generated)
    ├── _config/                          # Campaign-level configuration snapshot
    ├── _execution/                       # Execution metadata
    ├── _transient/                       # Intermediate/preprocessed data
    ├── _jobs/                            # Per-job artifacts (sysinfo, resource usage, logs)
    └── <config-name-1>/                  # One directory per configuration variant
    └── <config-name-2>/

Campaign-Level Directories

`_config/` — Configuration Snapshot

A copy of all input files used during execution. This folder can also be used to trigger another execution with the same configuration by running:

vast init <campaign-dir>/_config/<config-name>.vast
vast execution cluster run

The structure inside is domain-specific, but typically includes:

_config/
├── <name>.vast                            # The .vast configuration used
├── scenario.osc                           # OpenSCENARIO scenario file
├── analysis/                              # Jupyter notebooks for analysis
│   ├── analysis_run.ipynb
│   ├── analysis_config.ipynb
│   └── analysis_campaign.ipynb
└── <run-files defined within vast-config> # e.g. launch files, models, scripts, parameters

`_execution/` — Execution Metadata

_execution/
└── execution.yaml

Contains:

execution_time: ISO timestamp of when the execution started
robovast_version: Git commit hash of the robovast version used
runs: Number of runs per configuration
execution_type: cluster or local
image: Docker image with SHA digest
cluster_info: Node count, labels, CPU manager policies (cluster only)

`_transient/` — Intermediate Data

_transient/
├── configurations.yaml                   # Fully resolved configuration parameters
├── entrypoint.sh                         # Generated container entrypoint script
├── secondary_entrypoint.sh               # Generated secondary container entrypoint script
└── collect_sysinfo.py                    # System info collection script

configurations.yaml contains the fully resolved parameter values for every configuration variant, including internal computed fields like navigation path waypoints (_path), raster points (_raster_points), resolved file paths, and _variations (list of applied variation plugins with name, start time, duration, and any plugin-specific fields).

Configuration Directory

Each configuration variant gets its own directory:

<config-name>/
├── _config/
│   ├── config.yaml                       # Configuration identifier hashes
│   ├── scenario.config                   # Resolved parameter values (YAML)
│   ├── maps/                             # [navigation only]
│   │   ├── <name>.pgm                    # 2D occupancy grid image
│   │   └── <name>.yaml                   # Map metadata (resolution, origin, thresholds)
│   └── 3d-mesh/                          # [navigation only]
│       ├── <name>.stl                    # 3D environment mesh
│       └── <name>.stl.yaml               # Mesh metadata
├── _transient/                           # Per-config intermediate files
└── <run-number>/                         # 0, 1, 2, ... (one per run)

scenario.config contains the actual scenario parameter values used for this configuration, wrapped in a single key matching the scenario name:

test_scenario:
  growth_rate: 0.5
  initial_population: 50

Run Directory

Each run directory holds the scenario output for one configuration at one run number, plus a job symlink to that run’s job-level artifacts:

<run-number>/
├── test.xml                              # JUnit test result (pass/fail, duration)
├── job -> ../../_jobs/job-N              # symlink to this run's job artifacts (see below)
└── <test-specific files>                 # Domain-specific scenario output, e.g. out.csv,
                                          #   or a scenario-recorded rosbag2/ bag directory

Anything the scenario itself produces (test.xml, scenario-recorded rosbag2/, domain output) stays in the run directory. Infrastructure and monitoring artifacts (sysinfo.yaml, resource_usage_*.csv, the system log, and the entrypoint’s /rosout recording) belong to the job and live under _jobs/job-N/ — reachable via the job link, e.g. <run>/job/sysinfo.yaml (see Job Directory).

A common example of test-specific output is a scenario-recorded rosbag2/ directory (standard ROS 2 bag in MCAP storage, with a metadata.yaml listing recorded topics and message counts). It is present only when the scenario records a bag, and is distinct from the separate, job-level /rosout recording under _jobs/job-N/logs/.

`test.xml` — JUnit Test Result

Standard JUnit XML format with scenario execution results:

<testsuite errors="0" failures="0" name="scenario_execution" tests="1" time="49.03">
  <testcase classname="tests.scenario" name="test_scenario" time="49.03">
    <properties>
      <property name="start_time" value="1772634122.583653"/>
    </properties>
  </testcase>
</testsuite>

Job Directory

_jobs/job-N/ holds the artifacts of one job — the unit of dispatch (one Kubernetes Job, or one local docker compose run). With the default configs_per_job: 1 there is one job per run; with configs_per_job > 1 several configurations share a job (and therefore share these artifacts). Each run links to its job via <run>/job (e.g. <run>/job/sysinfo.yaml).

_jobs/job-N/
├── sysinfo.yaml                          # Hardware info (platform, CPU, memory) — stable
├── resource_usage_main.csv               # Main container CPU/memory over the job
├── resource_usage_<secondary>.csv        # Per secondary container [if multi-container]
└── logs/
    ├── system.log                        # Main container system log
    ├── system_<secondary>.log            # Secondary container log [if multi-container]
    └── rosout_bag/                       # /rosout recording [ROS mode]

resource_usage_*.csv files have columns timestamp, pid, name, cpu_usage, mem_usage (one per container). For a packed job these dynamic artifacts span the whole job; slicing them to a single configuration’s active time window is a planned post-processing step.

`metadata.yaml` — Campaign Metadata

Every campaign directory contains a metadata.yaml file that is automatically generated after postprocessing completes. It aggregates structural and domain-specific metadata about the entire campaign into a single file.

The file is produced by a three-phase pipeline:

Generic metadata — collected by MetadataGenerator (robovast.common.metadata). This includes configurations, test results (pass/fail, timing, output files, sysinfo), execution metadata, run files, and the scenario file reference.
Variation-plugin metadata — each variation plugin used during configuration generation can contribute additional metadata by overriding the collect_config_metadata classmethod defined on the Variation base class. For example, FloorplanGeneration overrides collect_config_metadata to load map and mesh YAML metadata from _config/. The variations field in each configuration entry lists all variation plugins that were applied, together with their execution timing (name, started_at as ISO timestamp, duration in seconds).
User-defined metadata processors — custom plugins registered under the robovast.metadata_processing entry-point group and configured in the .vast file (see below).

Example structure of metadata.yaml:

configurations:
  - name: config-1
    config:
      growth_rate: 0.5
      initial_population: 100
    config_files: []
    created_at: '2026-03-04T16:15:03.212496'
    variations:
      - name: FloorplanGeneration
        started_at: '2026-03-04T16:14:55.123456+00:00'
        duration: 3.217
      - name: PathVariationRandom
        started_at: '2026-03-04T16:14:58.340789+00:00'
        duration: 1.842
    test_results:
      - dir: config-1/0
        success: 'true'
        start_time: '2026-03-04T16:16:00+00:00'
        end_time: '2026-03-04T16:16:49'
        output_files:
          - config-1/0/sysinfo.yaml
          - config-1/0/logs/system.log
        sysinfo: { ... }
        postprocessing: {}
metadata: {}
run_files:
  - _config/files/growth_sim.py
scenario_file: scenario.osc
execution:
  execution_time: '2026-03-04T16:15:02'
  robovast_version: abc123
  runs: 2
  execution_type: cluster
  image: ghcr.io/example:latest

The metadata: block of the .vast file is passed through verbatim into metadata.yaml and is used to configure PROV-O generation (see below).

See Add Metadata Processing Plugin and Add Variation Plugin Metadata Hook for how to add custom metadata processing plugins and variation metadata hooks.

`metadata.prov.json` — PROV-O Provenance Graph

After metadata.yaml is written, RoboVAST automatically generates a W3C PROV-O provenance graph as <campaign-dir>/metadata.prov.json (JSON-LD format) and an optional metadata.pdf visualization (requires Graphviz dot).

The graph captures the full execution lineage of the campaign as a cyber-physical system test:

Software agents — RoboVAST and Scenery Builder with version info
Campaign activity — execution type, start time, number of runs
Scenario entities — abstract (.osc) and concrete per-configuration scenarios
Config-generation activity — links the .vast file to the generated configs
Per-run activities — success/failure, timing, sysinfo, output files
Domain-specific nodes — contributed by variation plugins (e.g. map/mesh entities for navigation, goal counts, obstacle counts); see Add PROV-O Provenance Hook to a Variation Plugin

Configuring the provenance graph

The metadata: section of the .vast file controls campaign-level provenance properties:

metadata:
  dataset_iri: https://purl.org/robovast/datasets/my-dataset/
  # Optional: list of CPS agents (robots, manipulators, etc.) involved
  # in the campaign.  Omit entirely for agent-free campaigns.
  agents:
    - name: turtlebot4
      type: robot
    - name: ur5
      type: manipulator

dataset_iri: Base IRI for the dataset namespace used in the provenance graph. All campaign, config, and run IRIs are constructed relative to this prefix. Defaults to https://purl.org/robovast/datasets/default/.
agents: List of PROV Agent nodes representing the physical systems under test (robots, manipulators, sensors, etc.). Each entry must have a name (used as the IRI fragment) and may carry arbitrary additional properties. If omitted, no agent nodes are added to the graph — suitable for software-only or simulation-only campaigns.

Domain-specific provenance nodes (e.g. navigation map/mesh entities) are contributed automatically by variation plugins that implement collect_prov_metadata; no manual configuration is required.

Postprocessing

Postprocessing transforms raw run output (e.g. ROS bags, custom binary files) into analysis-friendly formats (e.g. CSV). Commands are defined in the results_processing.postprocessing section of the .vast file and executed by plugins (see Add Postprocessing Command Plugin for how to write your own).

vast results postprocess [OPTIONS]

Options

-r, --results-dir PATH: Directory containing the run results (parent of campaign directories). When omitted the value configured with vast init is used.

-f, --force: Bypass the postprocessing cache and re-run all commands even if the results directory has not changed since the last postprocessing run.

-o, --override VAST_FILE: Use the given .vast file instead of the one stored in <campaign-name>-<timestamp>/_config/. See Using --override to Supply a Local .vast File for details.

Postprocessing is cached by a hash of the results directory. When the directory is unchanged the step is skipped automatically. Use --force (or -f) to bypass the cache, for example after updating a postprocessing script:

vast results postprocess --force

Publishing Results

Publication packages or distributes the results directory using plugins defined in the results_processing.publication section of the .vast file. Unlike postprocessing (which operates per campaign run folder), publication plugins receive the full results directory as input and are intended for tasks like creating zip archives for upload or hand-off.

vast results publish [OPTIONS]

Options

-r, --results-dir PATH: Directory containing the run results (parent of campaign directories). When omitted the value configured with vast init is used.

-o, --override VAST_FILE: Use the given .vast file instead of the one stored in <campaign-name>-<timestamp>/_config/.

-f, --force: Overwrite existing output files (e.g. zip archives) without prompting. Equivalent to setting overwrite: true on every publication plugin. Without this flag, plugins that find an existing output file will ask the user interactively (default answer: yes / overwrite).

Example:

# Publish using the project-configured results directory
vast results publish

# Publish and overwrite any existing archives without prompting
vast results publish --force

# Publish a specific results directory with an override config
vast results publish --results-dir /path/to/results --override my_project.vast

Listing Publication Plugins

vast results publish-commands

Lists all available publication plugins, their descriptions, and parameters. Useful for discovering which plugins can be used in the results_processing.publication section of the .vast file.

Merging Results

vast results merge-campaigns MERGED_CAMPAIGN_DIR [OPTIONS]

Merges campaign-directories with identical configs into one merged_campaign_dir. Groups campaign-directory/config-directory by config_identifier from config.yaml. Run folders (0, 1, 2, …) from all campaigns are renumbered and copied. Original campaign-directories are not modified.

Arguments

MERGED_CAMPAIGN_DIR: Target directory where the merged campaign will be written.

Options

-r, --results-dir PATH: Source directory containing campaign directories. When omitted the value configured with vast init is used.

Listing Postprocessing Plugins

vast results postprocess-commands

Lists all available postprocessing command plugins, their descriptions, and parameters. Useful for discovering which commands can be used in the results_processing.postprocessing section of the .vast file.

Using `--override` to Supply a Local `.vast` File

By default vast results postprocess reads the .vast configuration from the campaign snapshot stored in <results-dir>/<campaign-name>-<timestamp>/_config/<name>.vast. This snapshot is copied at execution time and may be out of date.

--override (short form -o) lets you point to any .vast file on disk, for example your current working copy:

# Use a local/updated .vast file
vast results postprocess --override my_project.vast

When to use ``–override``

You want to apply updated postprocessing scripts to existing results without triggering a new execution campaign.
The results were produced in a different directory and the campaign snapshot points to stale paths.
You want to bypass the snapshot and always use the latest .vast during iterative postprocessing development.

Note

When --override is supplied, the same .vast file is used for every campaign folder found under the results directory. The config directory of the override file (its parent folder) is used to resolve relative paths.

Results Processing

Output Structure

Top-Level Layout

Campaign-Level Directories

_config/ — Configuration Snapshot

_execution/ — Execution Metadata

_transient/ — Intermediate Data

Configuration Directory

Run Directory

test.xml — JUnit Test Result

Job Directory

metadata.yaml — Campaign Metadata

metadata.prov.json — PROV-O Provenance Graph

Postprocessing

Publishing Results

Listing Publication Plugins

Merging Results

Listing Postprocessing Plugins

Using --override to Supply a Local .vast File

`_config/` — Configuration Snapshot

`_execution/` — Execution Metadata

`_transient/` — Intermediate Data

`test.xml` — JUnit Test Result

`metadata.yaml` — Campaign Metadata

`metadata.prov.json` — PROV-O Provenance Graph

Using `--override` to Supply a Local `.vast` File