Clusters

class ducktape.cluster.cluster.Cluster

Bases: object

Interface for a cluster – a collection of nodes with login credentials. This interface doesn’t define any mapping of roles/services to nodes. It only interacts with some underlying system that can describe available resources and mediates reservations of those resources.

__init__()

all()

Return a ClusterSpec object describing all nodes.

alloc(cluster_spec)

Allocate some nodes.

Parameters:cluster_spec – A ClusterSpec describing the nodes to be allocated. Throws InsufficientResources:  If the nodes cannot be allocated. Returns:Allocated nodes spec

available()

Return a ClusterSpec object describing the currently available nodes.

do_alloc(cluster_spec)

Subclasses should implement actual allocation here.

Parameters:cluster_spec – A ClusterSpec describing the nodes to be allocated. Throws InsufficientResources:  If the nodes cannot be allocated. Returns:Allocated nodes spec

free(nodes)

Free the given node or list of nodes

used()

Return a ClusterSpec object describing the currently in use nodes.

class ducktape.cluster.vagrant.VagrantCluster(*args, make_remote_account_func=<function make_remote_account>, **kwargs)

Bases: ducktape.cluster.json.JsonCluster

An implementation of Cluster that uses a set of VMs created by Vagrant. Because we need hostnames that can be advertised, this assumes that the Vagrant VM’s name is a routeable hostname on all the hosts.

• If cluster_file is specified in the constructor’s kwargs (i.e. passed via command line argument –cluster-file) - If cluster_file exists on the filesystem, read cluster info from the file - Otherwise, retrieve cluster info via “vagrant ssh-config” from vagrant and write cluster info to cluster_file
• Otherwise, retrieve cluster info via “vagrant ssh-config” from vagrant

__init__(*args, make_remote_account_func=<function make_remote_account>, **kwargs)

class ducktape.cluster.localhost.LocalhostCluster(*args, **kwargs)

Bases: ducktape.cluster.cluster.Cluster

A “cluster” that runs entirely on localhost using default credentials. This doesn’t require any user configuration and is equivalent to the old defaults in cluster_config.json. There are no constraints on the resources available.

__init__(*args, **kwargs)

class ducktape.cluster.json.JsonCluster(cluster_json=None, *args, make_remote_account_func=<function make_remote_account>, **kwargs)

Bases: ducktape.cluster.cluster.Cluster

An implementation of Cluster that uses static settings specified in a cluster file or json-serializeable dict

__init__(cluster_json=None, *args, make_remote_account_func=<function make_remote_account>, **kwargs)

Initialize JsonCluster

JsonCluster can be initialized from:

• a json-serializeable dict
• a “cluster_file” containing json

Parameters:

• cluster_json – a json-serializeable dict containing node information. If cluster_json is None, load from file
• (optional) (cluster_file) – Overrides the default location of the json cluster file

Example json with a local Vagrant cluster:

{
  "nodes": [
    {
      "externally_routable_ip": "192.168.50.151",

      "ssh_config": {
        "host": "worker1",
        "hostname": "127.0.0.1",
        "identityfile": "/path/to/private_key",
        "password": null,
        "port": 2222,
        "user": "vagrant"
      }
    },
    {
      "externally_routable_ip": "192.168.50.151",

      "ssh_config": {
        "host": "worker2",
        "hostname": "127.0.0.1",
        "identityfile": "/path/to/private_key",
        "password": null,
        "port": 2223,
        "user": "vagrant"
      }
    }
  ]
}