Services

class ducktape.services.service.Service(context, num_nodes=None, cluster_spec=None, *args, **kwargs)

Bases: ducktape.template.TemplateRenderer

Service classes know how to deploy a service onto a set of nodes and then clean up after themselves.

They request the necessary resources from the cluster, configure each node, and bring up/tear down the service.

They also expose information about the service so that other services or test scripts can easily be configured to work with them. Finally, they may be able to collect and check logs/output from the service, which can be helpful in writing tests or benchmarks.

Services should generally be written to support an arbitrary number of nodes, even if instances are independent of each other. They should be able to assume that there won’t be resource conflicts: the cluster tests are being run on should be large enough to use one instance per service instance.

__init__(context, num_nodes=None, cluster_spec=None, *args, **kwargs)

Initialize the Service.

Note: only one of (num_nodes, cluster_spec) may be set.

Parameters:

• context – An object which has at minimum ‘cluster’ and ‘logger’ attributes. In tests, this is always a TestContext object.
• num_nodes – An integer representing the number of Linux nodes to allocate.
• cluster_spec – A ClusterSpec object representing the minimum cluster specification needed.

allocate_nodes()

Request resources from the cluster.

allocated

Return True iff nodes have been allocated to this service instance.

clean(**kwargs)

Clean up persistent state on each node - e.g. logs, config files etc. Subclasses must override clean_node.

clean_node(node, **kwargs)

Clean up persistent state on this node - e.g. service logs, configuration files etc.

close()

Release resources.

cluster

The cluster object from which this service instance gets its nodes.

free()

Free each node. This ‘deallocates’ the nodes so the cluster can assign them to other services.

get_node(idx)

ids presented externally are indexed from 1, so we provide a helper method to avoid confusion.

idx(node)

Return id of the given node. Return -1 if node does not belong to this service.

idx identifies the node within this service instance (not globally).

local_scratch_dir

This local scratch directory is created/destroyed on the test driver before/after each test is run.

logger

The logger instance for this service.

run()

Helper that executes run(), wait(), and stop() in sequence.

static run_parallel(*args)

Helper to run a set of services in parallel. This is useful if you want multiple services of different types to run concurrently, e.g. a producer + consumer pair.

service_id

Human-readable identifier (almost certainly) unique within a test run.

start(**kwargs)

Start the service on all nodes.

start_node(node, **kwargs)

Start service process(es) on the given node.

stop(**kwargs)

Stop service processes on each node in this service. Subclasses must override stop_node.

stop_node(node, **kwargs)

Halt service process(es) on this node.

wait(timeout_sec=600)

Wait for the service to finish. This only makes sense for tasks with a fixed amount of work to do. For services that generate output, it is only guaranteed to be available after this call returns.

wait_node(node, timeout_sec=None)

Wait for the service on the given node to finish. Return True if the node finished shutdown, False otherwise.

who_am_i(node=None)

Human-readable identifier useful for log messages.

class ducktape.services.background_thread.BackgroundThreadService(context, num_nodes=None, cluster_spec=None, *args, **kwargs)

Bases: ducktape.services.service.Service

__init__(context, num_nodes=None, cluster_spec=None, *args, **kwargs)

wait(timeout_sec=600)

Wait no more than timeout_sec for all worker threads to finish.

raise TimeoutException if all worker threads do not finish within timeout_sec