flux start [OPTIONS] [initial-program [args...]]


flux-start(1) launches a new Flux instance. By default, flux-start execs a single flux-broker(1) directly, which will attempt to use PMI to fetch job information and bootstrap a flux instance.

If a size is specified via --test-size, an instance of that size is to be started on the local host with flux-start as the parent.

A failure of the initial program (such as non-zero exit code) causes flux-start to exit with a non-zero exit code.


-o, --broker-opts=option_string

Add options to the message broker daemon, separated by commas.

-v, --verbose=[LEVEL]

This option may be specified multiple times, or with a value, to set a verbosity level. See VERBOSITY LEVELS below.

-X, --noexec

Don't execute anything. This option is most useful with -v.


Run brokers with Caliper profiling enabled, using a Caliper configuration profile named PROFILE. Requires a version of Flux built with --enable-caliper. Unless CALI_LOG_VERBOSITY is already set in the environment, it will default to 0 for all brokers.


(only with --test-size) Set the directory that will be used as the rundir directory for the instance. If the directory does not exist then it will be created during instance startup. If a DIR is not set with this option, a unique temporary directory will be created. Unless DIR was pre-existing, it will be removed when the instance is destroyed.


Wrap broker execution in a comma-separated list of arguments. This is useful for running flux-broker directly under debuggers or valgrind.

-s, --test-size=N

Launch an instance of size N on the local host.


Set FLUX_FAKE_HOSTNAME in the environment of each broker so that the broker can bootstrap from a config file instead of PMI. HOSTLIST is assumed to be in rank order. The broker will use the fake hostname to find its entry in the configured bootstrap host array.


After a broker exits, kill the other brokers after a timeout (default 20s).


Set the mode for the exit timeout. If set to leader, the exit timeout is only triggered upon exit of the leader broker, and the flux-start exit code is that of the leader broker. If set to any, the exit timeout is triggered upon exit of any broker, and the flux-start exit code is the highest exit code of all brokers. Default: any.


Set the start mode. If set to all, all brokers are started immediately. If set to leader, only the leader is started. Hint: in leader mode, use --setattr=broker.quorum=1 to let the initial program start before the other brokers are online. Default: all.


Set the directory to be used as the broker rundir instead of creating a temporary one. The directory must exist, and is not cleaned up unless --test-rundir-cleanup is also specified.


Recursively remove the directory specified with --test-rundir upon completion of flux-start.


Set the pmi clique mode, which determines how PMI_process_mapping is set in the PMI server used to bootstrap the brokers. If none, the mapping is not created. If single, all brokers are placed in one clique. If per-broker, each broker is placed in its own clique. Default: single.

-r, --recovery=[TARGET]

Start the rank 0 broker of an instance in recovery mode. If TARGET is a directory, treat it as a statedir from a previous instance. If TARGET is a file, treat it as an archive file from flux-dump(1). If TARGET is unspecified, assume the system instance is to be recovered. In recovery mode, any rc1 errors are ignored, broker peers are not allowed to connect, and resources are offline.


Run the broker with --config-path set to the default system instance configuration directory. This option is unnecessary if --recovery is specified without its optional argument. It may be required if recovering a dump from a system instance.


level 1 and above

Display commands before executing them.

level 2 and above

Trace PMI server requests (test mode only).


Launch an 8-way local Flux instance with an interactive shell as the initial program and all logs output to stderr:

flux start -s8 -o,--setattr=log-stderr-level=7

Launch an 8-way Flux instance within a slurm job, with an interactive shell as the initial program:

srun --pty -N8 flux start

Start the system instance rank 0 broker in recovery mode:

sudo -u flux flux start --recovery

Start a non-system instance in recovery mode:

flux start --recovery=/tmp/statedir


Flux: http://flux-framework.org