How to run

vast configuration

Manage test configuration.

vast configuration [OPTIONS] COMMAND [ARGS]...

generate

Generate test configurations and output files.

Creates all configurations and associated files in the configured results directory.

Requires project initialization with vast init first.

vast configuration generate [OPTIONS] OUTPUT_DIR

Arguments

OUTPUT_DIR: Required argument

variation-points

List possible variation points from the scenario files.

Shows all available variation points (scenario parameters) that can be varied in the scenarios as defined in the vast configuration file.

Requires project initialization with vast init first.

vast configuration variation-points [OPTIONS]

variation-types

List available variation types.

Shows all registered variation type entry points that can be used in the variations section of .vast configuration files.

vast configuration variation-types [OPTIONS]

list

List scenario configs without generating files.

This command shows all configs that would be generated from the configuration file without actually creating the output files.

Requires project initialization with vast init first.

vast configuration list [OPTIONS]

Options

--debug: Show internal values starting with _

vast execution

Execute scenarios locally or on a cluster.

Run scenario configurations either locally using Docker or on a Kubernetes cluster for distributed execution.

vast execution [OPTIONS] COMMAND [ARGS]...

local

Execute scenarios locally using Docker.

Run test configurations in Docker containers with bind mounts for configuration and output data.

Requires project initialization with vast init first.

vast execution local [OPTIONS] COMMAND [ARGS]...

prepare-run

Prepare run without executing.

Generates all necessary configuration files and a run.sh script for manual execution. This is useful for inspecting the generated configuration, debugging, or executing scenarios with custom modifications.

This command does NOT execute the scenario - it only prepares the files. Use vast execution local run for immediate execution.

Prerequisites: - Project initialized with vast init

Generated files in OUTPUT-DIR: - config/: Directory containing all scenario configuration files - run.sh: Executable shell script to run the scenario with Docker - Various temporary configuration files for the execution

After preparation, inspect the files in OUTPUT-DIR and execute manually cd OUTPUT-DIR; ./run.sh.

The run.sh script supports the same options as vast execution local run (–shell, –no-gui, –network-host, –output, –image).

vast execution local prepare-run [OPTIONS] OUTPUT_DIR

Options

-c, --config <config>: Run only a specific configuration by name

-r, --runs <runs>: Override the number of runs specified in the config

Arguments

OUTPUT_DIR: Required argument

run

Execute scenario configurations locally using Docker.

Runs scenario configurations in Docker containers with bind mounts for configuration and output data. By default, runs all configurations from the project configuration. GUI support is enabled by default (requires X11 server on host).

Prerequisites: - Docker must be installed and running - Project initialized with vast init - X11 server running on host (for GUI support, disable with --no-gui)

Output:: Results are written to the project results directory by default, or to a custom directory specified with --output.

vast execution local run [OPTIONS]

Options

-c, --config <config>: Run only a specific configuration by name

-r, --runs <runs>: Override the number of runs specified in the config

-o, --output <output>: Output directory (uses project results dir if not specified)

-s, --shell: Instead of running the scenario, login with shell

--no-gui: Disable host GUI support

--network-host: Use host network mode

-i, --image <image>: Use a custom Docker image

cluster

Execute scenarios on a Kubernetes cluster.

Run scenario configurations as Kubernetes jobs with bind mounts for configuration and output data.

Requires project initialization with vast init first.

vast execution cluster [OPTIONS] COMMAND [ARGS]...

cleanup

Clean up the Kubernetes cluster setup.

Removes the NFS server pod and service from the Kubernetes cluster by deleting the NFS manifest configuration.

This command can be run after completing all scenario executions to clean up cluster infrastructure resources (different from run-cleanup which only cleans up job pods).

If --cluster-config is not specified, it will automatically detect which cluster configuration was used during setup (from the project flag file). When specifying --cluster-config explicitly, pass -n <namespace> if the setup was done in a non-default namespace.

vast execution cluster cleanup [OPTIONS]

Options

-c, --cluster-config <config_name>: Cluster configuration plugin to use (auto-detects if not specified)

-n, --namespace <namespace>: Kubernetes namespace to clean up (required when using –cluster-config without prior setup)

-o, --option <options>: Cluster-specific option in key=value format (can be used multiple times)

download

Download result files from the cluster S3 (MinIO) server.

Downloads all test run results from the MinIO S3 server embedded in the robovast pod. Each run is stored in a separate S3 bucket (run-*) and downloaded into a subdirectory of the output directory.

Use --force to re-download runs that already exist locally.

Requires project initialization with vast init first (unless --output is specified).

vast execution cluster download [OPTIONS]

Options

-o, --output <output>: Directory where all runs will be downloaded (uses project results dir if not specified)

-f, --force: Force re-download even if files already exist locally

monitor

Monitor scenario execution jobs on the cluster.

Continuously displays how many jobs have finished (completed or failed), how many are currently running, and how many are still pending.

This is intended for monitoring jobs created by vast execution cluster run -d.

vast execution cluster monitor [OPTIONS]

Options

-i, --interval <interval>

Polling interval in seconds

Default:: 2.0

--once: Print job status once and exit

prepare-run

Prepare complete setup for manual deployment.

Generates all necessary files for cluster execution and writes them to the specified output directory.

The output directory will contain: - config/ directory with all scenario configurations - jobs/ directory with individual job manifest YAML files - all-jobs.yaml file with all jobs combined - copy_configs.py script to upload test configurations to the cluster - README.md with general execution instructions - Cluster-specific setup files (manifests, templates, README)

The generated package is self-contained and can be used to: 1. Set up the cluster infrastructure (NFS server, PVCs) 2. Upload configuration files to the cluster 3. Deploy and execute all scenario jobs

Cluster-specific options can be passed using –option key=value.

Requires project initialization with vast init first.

vast execution cluster prepare-run [OPTIONS] OUTPUT

Options

-c, --config <config>: Prepare only a specific config by name

-r, --runs <runs>: Override the number of runs specified in the config

-k, --cluster-config <cluster_config>: Override the cluster configuration specified in the config

-o, --option <options>: Cluster-specific option in key=value format (can be used multiple times)

Arguments

OUTPUT: Required argument

run

Execute scenarios on a Kubernetes cluster.

Deploys all test configurations (or a specific one) as Kubernetes jobs for distributed parallel execution.

Use –detach to exit immediately after creating jobs without waiting. Use ‘vast execution cluster run-cleanup’ to clean up jobs afterwards.

Requires project initialization with vast init first.

vast execution cluster run [OPTIONS]

Options

-c, --config <config>: Run only a specific configuration by name

-r, --runs <runs>: Override the number of runs specified in the config

-d, --detach: Exit after creating jobs without waiting for completion

run-cleanup

Clean up jobs and pods from a cluster run.

Removes all scenario execution jobs and their associated pods. This is useful after running with –detach to clean up resources once jobs have completed.

Usage: vast execution cluster run-cleanup

vast execution cluster run-cleanup [OPTIONS]

setup

Set up the Kubernetes cluster for execution.

Deploys a MinIO S3 server in the Kubernetes cluster. The server is used to store run configurations and results for individual scenario execution jobs.

This command should be run once before executing scenarios on the cluster for the first time.

If the cluster is already set up, this command will exit with an error. Run ‘vast execution cluster cleanup’ first to clean up the existing setup, or use --force to force re-setup.

Use --list to see available cluster configuration plugins.

Cluster-specific options can be passed using --option key=value.

vast execution cluster setup [OPTIONS] [CLUSTER_CONFIG]

Options

--list: List available cluster configuration plugins

-n, --namespace <namespace>

Kubernetes namespace for execution (used by cluster run)

Default:: default

-o, --option <options>: Cluster-specific option in key=value format (can be used multiple times)

-f, --force: Force re-setup even if cluster is already set up

Arguments

CLUSTER_CONFIG: Optional argument

vast analysis

Analyze test results and generate reports.

Tools for visualizing and analyzing scenario execution results.

vast analysis [OPTIONS] COMMAND [ARGS]...

postprocess

Run postprocessing commands on test results.

Executes postprocessing commands defined in the configuration file’s analysis.postprocessing section. Postprocessing is skipped if the result-directory is unchanged, unless –force is specified.

Requires project initialization with vast init first (unless --results-dir is specified).

vast analysis postprocess [OPTIONS]

Options

-r, --results-dir <results_dir>: Directory containing test results (uses project results dir if not specified)

-f, --force: Force postprocessing even if results directory is unchanged (bypasses caching)

gui

Launch the graphical test results analyzer.

Opens a GUI application for interactive exploration and visualization of test results. Automatically runs postprocessing before launching the GUI.

Requires project initialization with vast init first (unless --results-dir is specified).

vast analysis gui [OPTIONS]

Options

-r, --results-dir <results_dir>: Directory containing test results (uses project results dir if not specified)

-f, --force: Force postprocessing even if results directory is unchanged (bypasses caching)