flux.subprocess module

class flux.subprocess.Subprocess(pid: int, rank: int, state: str, label: str, cmd: str, bg: bool = False, waitable: bool = False, attached: bool = False)

Bases: object

Class representing a Flux subprocess object as returned from rexec.list

attached: bool = False
bg: bool = False
cmd: str
property flags

comma-separated list of bg and waitable, as applicable

label: str
pid: int
rank: int
state: str
property statex

state with a ps(1) style "+" appended while a client is attached

waitable: bool = False
class flux.subprocess.SubprocessBackgroundRexecRPC(handle, *args, **kwargs)

Bases: RPC

RPC object returned from rexec_bg()

get_pid()

Return process id of remote process

get_rank()

Return rank on which remote process was launched

class flux.subprocess.SubprocessListRPC(flux_handle, topic, payload=None, nodeid=flux.constants.FLUX_NODEID_ANY, flags=0)

Bases: RPC

get_processes()

Get a list of Subprocess objects from an rexec.list response.

Returns a list of Subprocess objects, one for each remote process.

class flux.subprocess.SubprocessWaitRPC(flux_handle, topic, payload=None, nodeid=flux.constants.FLUX_NODEID_ANY, flags=0)

Bases: RPC

RPC object returned from wait()

get_output()

Return output retained by the server for a background subprocess.

The server may retain a bounded, possibly incomplete amount of the most recent output produced by a waitable background subprocess. Each element is an RFC 42 I/O object (a dict with keys stream, rank, data, and optionally encoding). The list is empty if the server retained no output.

Returns

retained I/O objects in the order produced

Return type

list

get_status()

Return raw wait status from remote process

Returns

wait status suitable for use with os.WIFEXITED(),

os.WEXITSTATUS(), etc.

Return type

int

property stderr

Retained standard error as bytes.

This concatenates the data of all retained stderr I/O objects, matching the familiar interface of subprocess.CompletedProcess. The result may be empty if the server retained no stderr.

Returns

retained standard error

Return type

bytes

property stdout

Retained standard output as bytes.

This concatenates the data of all retained stdout I/O objects, matching the familiar interface of subprocess.CompletedProcess. The result may be empty if the server retained no stdout.

Returns

retained standard output

Return type

bytes

flux.subprocess.command_create(command, label=None, cwd=None, env=None, opts=None)

Create an RFC 42 command object

Parameters

  • command (list) -- command line to execute

  • label (str) -- optional process label

  • cwd (str) -- current working directory (default: current directory)

  • env (dict) -- environment variables (default: copy of current environment)

  • opts (dict) -- additional options (default: empty dict)

Returns

RFC 42 command object suitable for use with subprocess

execution RPCs

Return type

dict

flux.subprocess.kill(handle, signum=15, pid=None, label=None, service='rexec', nodeid=flux.constants.FLUX_NODEID_ANY, sign=None)

Kill a subprocess

Kill a subprocess running on rank nodeid using either pid or label.

Parameters

  • handle (flux.Flux) -- Flux handle

  • signum (int) -- signal to send (default: SIGTERM)

  • pid (int) -- remote subprocess id

  • label (str) -- remote process label

  • service (str) -- remote service to use (default: "rexec")

  • nodeid (int) -- remote nodeid (default: FLUX_NODEID_ANY)

  • sign (bool or None) -- Sign the request with RFC 42 credentials. If None (default), signing is enabled automatically when the instance owner differs from the current userid.

Returns

flux.rpc.RPC

Raises

  • ValueError -- if neither or both of pid and label are specified

  • OSError -- on RPC failure

flux.subprocess.list(handle, service='rexec', nodeid=flux.constants.FLUX_NODEID_ANY, sign=None)

Get current subprocess list from a remote rank

Parameters

  • handle (flux.Flux) -- Flux handle

  • service (str) -- rexec service to use (default: "rexec")

  • nodeid (int) -- nodeid to target (default: FLUX_NODEID_ANY)

  • sign (bool or None) -- Sign the request with RFC 42 credentials. If None (default), signing is enabled automatically when the instance owner differs from the current userid.

Returns

SubprocessListRPC

flux.subprocess.rexec_bg(handle, command, cwd=None, env=None, label=None, waitable=False, service='rexec', nodeid=flux.constants.FLUX_NODEID_ANY, sign=None)

Run a process in the background

Parameters

  • handle (flux.Flux) -- Flux handle

  • command (list) -- command to run

  • cwd (str) -- current working directory of remote process. Default: use the cwd of the current process.

  • env (dict) -- environment of remote process. Default: use the current environment.

  • label (str) -- Give the remote process an RFC 42 label.

  • waitable (bool) -- Create the remote process with the waitable flag. Allows use of wait() function.

  • service (str) -- Use an alternate service (default: rexec)

  • nodeid (int) -- Target a different node (default: FLUX_NODEID_ANY)

  • sign (bool or None) -- Sign the request with RFC 42 credentials. If None (default), signing is enabled automatically when the instance owner differs from the current userid.

Returns

RPC object with get_pid()

and get_rank() methods

Return type

SubprocessBackgroundRexecRPC

Raises

OSError -- on failure to invoke remote process

flux.subprocess.wait(handle, pid=None, label=None, service='rexec', nodeid=flux.constants.FLUX_NODEID_ANY, sign=None)

Wait on a remote waitable process

Parameters

  • handle (flux.Flux) -- Flux handle

  • pid (int) -- remote subprocess id

  • label (str) -- remote process label

  • service (str) -- remote service to use (default: "rexec")

  • nodeid (int) -- remote nodeid (default: FLUX_NODEID_ANY)

  • sign (bool or None) -- Sign the request with RFC 42 credentials. If None (default), signing is enabled automatically when the instance owner differs from the current userid.

Returns

SubprocessWaitRPC

Raises

  • ValueError -- if neither or both of pid and label are specified

  • OSError -- on RPC failure