Run Tests¶

Running Tests¶

ducktape discovers and runs tests in the path provided, here are some ways to run tests:

ducktape <relative_path_to_testdirectory>               # e.g. ducktape dir/tests
ducktape <relative_path_to_file>                        # e.g. ducktape dir/tests/my_test.py
ducktape <path_to_test>[::SomeTestClass]                # e.g. ducktape dir/tests/my_test.py::TestA
ducktape <path_to_test>[::SomeTestClass[.test_method]]  # e.g. ducktape dir/tests/my_test.py::TestA.test_a

Options¶

To see a complete listing of options run:

ducktape --help

Discover and run your tests

usage: ducktape [-h] [--collect-only] [--collect-num-nodes] [--debug]
                [--config-file CONFIG_FILE] [--compress] [--cluster CLUSTER]
                [--default-num-nodes DEFAULT_NUM_NODES]
                [--cluster-file CLUSTER_FILE] [--results-root RESULTS_ROOT]
                [--exit-first] [--no-teardown] [--version]
                [--parameters PARAMETERS] [--globals GLOBALS]
                [--max-parallel MAX_PARALLEL] [--repeat REPEAT]
                [--subsets SUBSETS] [--subset SUBSET]
                [--historical-report HISTORICAL_REPORT] [--sample SAMPLE]
                [--fail-bad-cluster-utilization]
                [--test-runner-timeout TEST_RUNNER_TIMEOUT]
                [--ssh-checker-function SSH_CHECKER_FUNCTION [SSH_CHECKER_FUNCTION ...]]
                [--deflake DEFLAKE]
                [test_path [test_path ...]]

Required Arguments

`test_path=['/home/docs/checkouts/readthedocs.org/user_builds/ducktape/checkouts/0.7.x/docs']`
	one or more space-delimited strings indicating where to search for tests.

Optional Arguments

`--collect-only=False`
	display collected tests, but do not run.
`--collect-num-nodes=False`
	display total number of nodes requested by all tests, but do not run anything.
`--debug=False`	pipe more verbose test output to stdout.
`--config-file="~/.ducktape/config"`
	path to project-specific configuration file.
`--compress=False`
	compress remote logs before collection.
`--cluster="ducktape.cluster.vagrant.VagrantCluster"`
	cluster class to use to allocate nodes for tests.
`--default-num-nodes`
	Global hint for cluster usage. A test without the @cluster annotation will default to this value for expected cluster usage.
`--cluster-file`	path to a json file which provides information needed to initialize a json cluster. The file is used to read/write cached cluster info if cluster is ducktape.cluster.vagrant.VagrantCluster.
`--results-root="./results"`
	path to custom root results directory. Running ducktape with this root specified will result in new test results being stored in a subdirectory of this root directory.
`--exit-first=False`
	exit after first failure
`--no-teardown=False`
	don’t kill running processes or remove log files when a test has finished running. This is primarily useful for test developers who want to interact with running services after a test has run.
`--version=False`
	display version
`--parameters`	inject these arguments into the specified test(s). Specify parameters as a JSON string.
`--globals`	user-defined globals go here. This can be a file containing a JSON object, or a string representing a JSON object.
`--max-parallel=1`
	Upper bound on number of tests run simultaneously.
`--repeat=1`	Use this flag to repeat all discovered tests the given number of times.
`--subsets=1`	Number of subsets of tests to statically break the tests into to allow for parallel execution without coordination between test runner processes.
`--subset=0`	Which subset of the tests to run, based on the breakdown using the parameter for –subsets
`--historical-report`
	URL of a JSON report file containing stats from a previous test run. If specified, this will be used when creating subsets of tests to divide evenly by total run time instead of by number of tests.
`--sample`	The size of a random test sample to run
`--fail-bad-cluster-utilization=False`
	Fail a test if the cluster node utilization does not match the cluster node usage.
`--test-runner-timeout=1800000`
	Amount of time in milliseconds between test communicating between the test runner before a timeout error occurs. Default is 30 minutes
`--ssh-checker-function`
	Python module path(s) to a function that takes an exception and a remote account that will be called when an ssh error occurs, this can give some validation or better logging when an ssh error occurs. Specify any number of module paths after this flag to be called.
`--deflake=1`	the number of times a failed test should be ran in total (including its initial run) to determine flakyness. When not present, deflake will not be used, and a test will be marked as either passed or failed. When enabled tests will be marked as flaky if it passes on any of the reruns

Configuration File¶

You can configure options in three locations: on the command line (highest priority), in a user configuration file in ~/.ducktape/config, and in a project-specific configuration <project_dir>/.ducktape/config (lowest priority). Configuration files use the same syntax as command line arguments and may split arguments across multiple lines:

--debug
--exit-first
--cluster=ducktape.cluster.json.JsonCluster

Output¶

Test results go in results/<session_id>.<session_id> which looks like <date>--<test_number>. For example: results/2015-03-28--002

ducktape does its best to group test results and log files in a sensible way. The output directory is structured like so:

<session_id>
    session_log.info
    session_log.debug
    report.txt   # Summary report of all tests run in this session
    report.html  # Open this to see summary report in a browser
    report.css

    <test_class_name>
        <test_method_name>
            test_log.info
            test_log.debug
            report.txt   # Report on this single test
            [data.json]  # Present if the test returns data

            <service_1>
                <node_1>
                    some_logs
                <node_2>
                    some_logs
    ...

To see an example of the output structure, go here and click on one of the details links.