Remote Account

class ducktape.cluster.remoteaccount.RemoteAccount(ssh_config, externally_routable_ip=None, logger=None, ssh_exception_checks=[])

Bases: ducktape.utils.http_utils.HttpMixin

RemoteAccount is the heart of interaction with cluster nodes, and every allocated cluster node has a reference to an instance of RemoteAccount.

It wraps metadata such as ssh configs, and provides methods for file system manipulation and shell commands.

Each operating system has its own RemoteAccount implementation.

__init__(ssh_config, externally_routable_ip=None, logger=None, ssh_exception_checks=[])

alive(pid)

Return True if and only if process with given pid is alive.

close()

Close/release any outstanding network connections to remote account.

copy_between(src, dest, dest_node)

Copy src to dest on dest_node

Parameters:

• src – Path to the file or directory we want to copy
• dest – The destination path
• dest_node – The node to which we want to copy the file/directory

Note that if src is a directory, this will automatically copy recursively.

java_pids(match)

Get all the Java process IDs matching ‘match’.

Parameters:match – The AWK expression to match

kill_java_processes(match, clean_shutdown=True, allow_fail=False)

Kill all the java processes matching ‘match’.

Parameters:

• match – The AWK expression to match
• clean_shutdown – True if we should shut down cleanly with SIGTERM; false if we should shut down with SIGKILL.
• allow_fail – True if we should throw exceptions if the ssh commands fail.

monitor_log(log)

Context manager that returns an object that helps you wait for events to occur in a log. This checks the size of the log at the beginning of the block and makes a helper object available with convenience methods for checking or waiting for a pattern to appear in the log. This will commonly be used to start a process, then wait for a log message indicating the process is in a ready state.

See LogMonitor for more usage information.

remove(path, allow_fail=False)

Remove the given file or directory

wait_for_http_service(port, headers, timeout=20, path='/')

Wait until this service node is available/awake.

class ducktape.cluster.remoteaccount.LogMonitor(acct, log, offset)

Bases: object

Helper class returned by monitor_log. Should be used as:

with remote_account.monitor_log("/path/to/log") as monitor:
    remote_account.ssh("/command/to/start")
    monitor.wait_until("pattern.*to.*grep.*for", timeout_sec=5)

to run the command and then wait for the pattern to appear in the log.

__init__(acct, log, offset)

wait_until(pattern, **kwargs)

Wait until the specified pattern is found in the log, after the initial offset recorded when the LogMonitor was created. Additional keyword args are passed directly to ducktape.utils.util.wait_until

class ducktape.cluster.linux_remoteaccount.LinuxRemoteAccount(*args, **kwargs)

Bases: ducktape.cluster.remoteaccount.RemoteAccount

__init__(*args, **kwargs)

get_external_accessible_network_devices()

gets the subset of devices accessible through an external conenction

get_network_devices()

Utility to get all network devices on a linux account

local

Returns True if this ‘remote’ account is probably local. This is an imperfect heuristic, but should work for simple local testing.

class ducktape.cluster.windows_remoteaccount.WindowsRemoteAccount(*args, **kwargs)

Bases: ducktape.cluster.remoteaccount.RemoteAccount

Windows remote accounts are currently only supported in EC2. See _setup_winrm() for how the WinRM password is fetched, which is currently specific to AWS.

The Windows AMI needs to also have an SSH server running to support SSH commands, SCP, and rsync.

__init__(*args, **kwargs)