flux exec [--noinput] [--label-io] [—dir=DIR] [--rank=IDSET] [--verbose] COMMAND...


flux exec remotely executes one or more copies of COMMAND, similar to pdsh(1). It bypasses the scheduler and is intended for launching administrative commands or tool daemons, not for launching parallel jobs. For that, see flux-run(1).

By default, COMMAND runs across all flux-broker(1) processes. If the --jobid option is specified, the commands are run across a job's flux-shell(1) processes. Normally this means that one copy of COMMAND is executed per node, but in unusual cases it could mean more (e.g. if the Flux instance was started with multiple brokers per node).

Standard output and standard error of the remote commands are captured and combined on the flux exec standard output and standard error. Standard input of flux exec is captured and broadcast to standard input of the remote commands.

On receipt of SIGINT and SIGTERM signals, flux exec forwards the received signal to the remote processes. When standard input of flux exec is a terminal, Control-C may be used to send SIGINT. Two of those in short succession can force flux exec to exit in the event that remote processes are hanging.


-l, --label-io

Label lines of output with the source broker RANK. This option is not affected by --jobid.

-n, --noinput

Do not attempt to forward stdin. Send EOF to remote process stdin.

-d, --dir=DIR

Set the working directory of remote COMMAND to DIR. The default is to propagate the current working directory of flux-exec(1).

-r, --rank=IDSET

Target specific ranks, where IDSET is a set of zero-origin node ranks in RFC 22 format. If --jobid is specified, the ranks are interpreted as an index into the list of nodes assigned to the job. Otherwise, they refer to the nodes assigned to the Flux instance.

The default is to target all ranks. As a special case, --rank=all is accepted and behaves the same as the default.

-x, --exclude=IDSET

Exclude specific ranks. IDSET is as described in --rank.

-j, --jobid=JOBID

Run COMMAND on the nodes allocated to JOBID instead of the nodes assigned to the Flux instance.

This uses the exec service embedded in flux-shell(1) rather than flux-broker(1).

The interpretation of --rank and --exclude is adjusted as noted in their descriptions. For example, flux exec -j ID -r 0 will run only on the first node assigned to JOBID, and flux exec -j ID -x 0 will run on all nodes assigned to JOBID except the first node.

This option is only available when the job owner is the same as the Flux instance owner.

-v, --verbose

Run with more verbosity.

-q, --quiet

Suppress extraneous output (e.g. per-rank error exit status).


Prepend the full path to flux-imp run to COMMAND. This option is mostly meant for testing or as a convenience to execute a configured prolog or epilog command under the IMP. Note: When this option is used, or if flux-imp is detected as the first argument of COMMAND, flux exec will use flux-imp kill to signal remote commands instead of the normal builtin subprocess signaling mechanism.


In a multi-user flux instance, access to the rank 0 broker execution service is restricted to requests that originate from the local broker. Therefore, flux exec (without --jobid) must be run from the rank 0 broker if rank 0 is included in the target IDSET.


In the case that all processes are successfully launched, the exit status of flux exec is the largest of the remote process exit codes.

If a non-existent rank is targeted, flux exec will return with code 68 (EX_NOHOST from sysexits.h).

If one or more remote commands are terminated by a signal, then flux exec exits with exit code 128+signo.


Flux: http://flux-framework.org

Flux RFC: https://flux-framework.readthedocs.io/projects/flux-rfc


22/Idset String Representation