Command Line Interface¶

This documents each of the command line programs provided with Cosmic Ray. You can also get help on the command line by passed –help to any command.

`cosmic-ray`¶

This primary program provided by Cosmic Ray is, unsurprisingly, cosmic-ray. This program initializes sessions, performs mutations, and executes tests.

cosmic-ray¶

Mutation testing for Python3

cosmic-ray [OPTIONS] COMMAND [ARGS]...

Options

--verbosity <verbosity>¶

The logging level to use.

Options:: CRITICAL | DEBUG | ERROR | FATAL | INFO | WARNING

--version¶: Show the version and exit.

apply¶

Apply the specified mutation to the files on disk. This is primarily a debugging tool.

cosmic-ray apply [OPTIONS] MODULE_PATH OPERATOR OCCURRENCE

Arguments

MODULE_PATH¶: Required argument

OPERATOR¶: Required argument

OCCURRENCE¶: Required argument

baseline¶

Runs a baseline execution that executes the test suite over unmutated code.

If --session-file is provided, the session used for baselining is stored in that file. Otherwise, the session is stored in a temporary file which is deleted after the baselining.

Exits with 0 if the job has exited normally, otherwise 1.

cosmic-ray baseline [OPTIONS] CONFIG_FILE

Options

--session-file <session_file>¶: Path to session file. If not provided, a temp file is used.

Arguments

CONFIG_FILE¶: Required argument

distributors¶

List the available distributor plugins.

cosmic-ray distributors [OPTIONS]

dump¶

JSON dump of session data. This output is typically run through other programs to produce reports.

Each line of output is a list with two elements: a WorkItem and a WorkResult, both JSON-serialized. The WorkResult can be null, indicating a WorkItem with no results.

cosmic-ray dump [OPTIONS] SESSION_FILE

Arguments

SESSION_FILE¶: Required argument

exec¶

Perform the remaining work to be done in the specified session. This requires that the rest of your mutation testing infrastructure (e.g. worker processes) are already running.

cosmic-ray exec [OPTIONS] CONFIG_FILE SESSION_FILE

Arguments

CONFIG_FILE¶: Required argument

SESSION_FILE¶: Required argument

http-worker¶

Run an HTTP worker for the ‘http’ distributor.

cosmic-ray http-worker [OPTIONS]

Options

--port <port>¶: The port on which to listen for requests

--path <path>¶: Path to Unix domain socket on which to listen for requests

init¶

Initialize a mutation testing session from a configuration. This primarily creates a session - a database of “work to be done” - which describes all of the mutations and test runs that need to be executed for a full mutation testing run. The configuration specifies the top-level module to mutate, the tests to run, and how to run them.

This command doesn’t actually run any tests. Instead, it scans the modules-under-test and simply generates the work order which can be executed with other commands.

cosmic-ray init [OPTIONS] CONFIG_FILE SESSION_FILE

Options

--force¶: Re-initialize the session even if it already contains results

Arguments

CONFIG_FILE¶: Required argument

SESSION_FILE¶: Required argument

mutate-and-test¶

Run a worker process which performs a single mutation and test run. Each worker does a minimal, isolated chunk of work: it mutates the <occurrence>-th instance of <operator> in <module-path>, runs the test suite defined in the configuration, prints the results, and exits.

Normally you won’t run this directly. Rather, it will be launched by an distributor. However, it can be useful to run this on its own for testing and debugging purposes.

cosmic-ray mutate-and-test [OPTIONS] MODULE_PATH OPERATOR OCCURRENCE
                           TEST_COMMAND

Options

--keep-stdout¶: Do not squelch output.

Arguments

MODULE_PATH¶: Required argument

OPERATOR¶: Required argument

OCCURRENCE¶: Required argument

TEST_COMMAND¶: Required argument

new-config¶

Create a new config file.

cosmic-ray new-config [OPTIONS] CONFIG_FILE

Arguments

CONFIG_FILE¶: Required argument

operators¶

List the available operator plugins.

cosmic-ray operators [OPTIONS]

Concurrency¶

Note that most Cosmic Ray commands can be safely executed while exec is running. One exception is init since that will rewrite the work manifest.

For example, you can run cr-report on a session while that session is being executed. This will tell you what progress has been made.

`cr-html`¶

cr-html¶

Print an HTML formatted report of test results.

cr-html [OPTIONS] SESSION_FILE

Options

--only-completed, --not-only-completed¶

--skip-success, --include-success¶

--hide-skipped, --show-skipped¶

Arguments

SESSION_FILE¶: Required argument

`cr-report`¶

cr-report¶

Print a nicely formatted report of test results and some basic statistics.

cr-report [OPTIONS] SESSION_FILE

Options

--show-output, --no-show-output¶: Display output of test executions

--show-diff, --no-show-diff¶: Display diff of mutants

--show-pending, --no-show-pending¶: Display results for incomplete tasks

--surviving-only, --all-mutations¶: Only display completed work items whose tests survived

Arguments

SESSION_FILE¶: Required argument

Use --surviving-only alongside --show-diff (and/or --show-output) to focus the detailed listings on mutants whose tests survived, e.g. cr-report session.sqlite --show-diff --surviving-only.

`cr-badge`¶

cr-badge¶

Generate badge file.

cr-badge [OPTIONS] CONFIG_FILE BADGE_FILE SESSION_FILE

Arguments

CONFIG_FILE¶: Required argument

BADGE_FILE¶: Required argument

SESSION_FILE¶: Required argument

`cr-rate`¶

cr-rate¶

Calculate the survival rate of a session.

cr-rate [OPTIONS] SESSION_FILE

Options

--estimate, --no-estimate¶: Print the lower bound, estimate and upper bound of survival rate

--confidence <confidence>¶

Specify the confidence levels for estimates

Options:: 80.0 | 90.0 | 95.0 | 98.0 | 99.0 | 99.5 | 99.8 | 99.9

--fail-over <fail_over>¶: Exit with a non-zero code if the survival rate is larger than <max_value> or the calculated confidence interval is above the <max_value> (if –estimate is used). Specified as percentage.

Arguments

SESSION_FILE¶: Required argument

`cr-xml`¶

cr-xml¶

Print an XML formatted report of test results for continuous integration systems

cr-xml [OPTIONS] SESSION_FILE

Arguments

SESSION_FILE¶: Required argument

`cr-http-workers`¶

cr-http-workers¶

A tool for launching HTTP workers and executing a session using them.

This reads the ‘distributor.http.worker-urls’ field of a config to see where workers are expected to be running. For each worker, it makes a clone of the git repository that’s going to be tested, optionally changing to a directory under the root of the clone before starting the worker. It then starts the workers with the correct options to provide the configured URLs.

cr-http-workers [OPTIONS] CONFIG_FILE REPO_URL

Options

--location <location>¶: The relative path under a repo clone at which to run the worker

Arguments

CONFIG_FILE¶: Required argument

REPO_URL¶: Required argument

`cr-filter-operators`¶

TODO

`cr-filter-pragma`¶

TODO

`cr-filter-git`¶

TODO

Command Line Interface¶

`cosmic-ray`¶

cosmic-ray¶

apply¶

baseline¶

distributors¶

dump¶

exec¶

http-worker¶

init¶

mutate-and-test¶

new-config¶

operators¶

Concurrency¶

`cr-html`¶

cr-html¶

`cr-report`¶

cr-report¶

`cr-badge`¶

cr-badge¶

`cr-rate`¶

cr-rate¶

`cr-xml`¶

cr-xml¶

`cr-http-workers`¶

cr-http-workers¶

`cr-filter-operators`¶

`cr-filter-pragma`¶

`cr-filter-git`¶

Cosmic Ray

Navigation

Related Topics

Command Line Interface¶

cosmic-ray¶

cosmic-ray¶

apply¶

baseline¶

distributors¶

dump¶

exec¶

http-worker¶

init¶

mutate-and-test¶

new-config¶

operators¶

Concurrency¶

cr-html¶

cr-html¶

cr-report¶

cr-report¶

cr-badge¶

cr-badge¶

cr-rate¶

cr-rate¶

cr-xml¶

cr-xml¶

cr-http-workers¶

cr-http-workers¶

cr-filter-operators¶

cr-filter-pragma¶

cr-filter-git¶

`cosmic-ray`¶

`cr-html`¶

`cr-report`¶

`cr-badge`¶

`cr-rate`¶

`cr-xml`¶

`cr-http-workers`¶

`cr-filter-operators`¶

`cr-filter-pragma`¶

`cr-filter-git`¶