flux-kvs(1)
SYNOPSIS
DESCRIPTION
The Flux key-value store (KVS) is a simple, distributed data storage service used a building block by other Flux components. flux kvs is a command line utility that operates on the KVS.
The Flux KVS stores values under string keys. The keys are hierarchical, using "." as a path separator, analogous to "/" separated UNIX file paths. A single "." represents the root directory of the KVS.
The KVS is distributed among the broker ranks of a Flux instance. Rank 0 is the leader, and other ranks are caching followers. All writes are flushed to the leader during a commit operation. Data is stored in a hash tree such that every commit results in a new root hash. Each new root hash is multicast across the Flux instance. When followers update their root hash, they atomically update their view to match the leader. There may be a delay after a commit while old data is served on a follower that has not yet updated its root hash, thus the Flux KVS cache is "eventually consistent". Followers expire cache data after a period of disuse, and fault in new data through their parent in the overlay network.
The primary KVS namespace is only accessible to the Flux instance owner. Other namespaces may be created and assigned to guest users. Although the cache is shared across namespaces, each has an independent root directory, which permits commits in multiple namespaces to complete in parallel.
COMMANDS
The flux kvs sub-commands and their arguments are described below.
The following options are common to most sub-commands:
- -h, --help
Display help on this sub-command.
- -N, --namespace=NAME
Specify an alternate namespace. By default, the primary KVS namespace or the value of
FLUX_KVS_NAMESPACEis used.
get
Retrieve the value stored under key and print it on standard output.
It is an error if key does not exist, unless :option:--waitcreate is
specified. It is an error if key is a directory.
If multiple key arguments are specified, their values are concatenated.
A newline is appended to each value, unless --raw is specified or
the value is zero length.
- -r, --raw
Display value without a newline.
- -t, --treeobj
Display RFC 11 tree object.
- -a, --at=TREEOBJ
Perform the lookup relative to a directory reference in RFC 11 tree object format.
- -l, --label
Add key= prefix to output.
- -W, --waitcreate
If the key does not exist, wait until it does, then return its value.
- -w, --watch
Display the initial value for key, then watch for changes and display each new value until interrupted or
--count=Nis reached.
- -u, --uniq
With
--watch, suppress output when value is the same, even if the watched key was the target of a KVS commit.
- -f, --full
With
--watch, monitor key changes with more complete accuracy.By default, only a direct write to a key is monitored, thus changes that occur indirectly could be missed, such as when the parent directory is replaced. The
--fulloption ensures these changes are reported as well, at greater overhead.
put
Set key to value. If key exists, the current value is overwritten. If multiple key=value pairs are specified, they are sent as one commit and succeed or fail atomically.
- -O, --treeobj-root
After the commit has completed, display the new root directory reference in RFC 11 tree object format.
- -b, --blobref
After the commit has completed, display the new root directory reference as a single blobref.
- -s, --sequence
After the commit has completed, display the new root sequence number or "version".
- -r, --raw
Store value as-is, without adding NUL termination. If value is
-, read it from standard input. value may include embedded NUL bytes.
- -t, --treeobj
Interpret value as an RFC 11 tree object to be stored directly. If value is
-, read it from standard input.
- -n, --no-merge
Set the
NO_MERGEflag on the commit to ensure it is not combined with other commits. The KVS normally combines contemporaneous commits to save work, but since commits succeed or fail atomically, this a commit could fail due to collateral damage. This option prevents that from happening.
- -A, --append
Append value to key's existing value, if any, instead of overwriting it.
- -S, --sync
After the commit has completed, flush pending content and checkpoints to disk. Commits normally complete with data in memory, which ensures a subsequent flux kvs get would receive the updated value, but does not ensure it persists across a Flux instance crash. This can be used to ensure critical data has been written to non-volatile storage.
dir
Display all keys and their values under the directory key. Values that
are too long to fit the terminal width are truncated with "..." appended.
This command fails if key does not exist or is not a directory.
If key is not provided, . (root of the namespace) is assumed.
- -R, --recursive
Recursively display keys under subdirectories.
- -d, --directory
List directory entries but not values.
- -w, --width=N
Truncate values to fit in N columns instead of the terminal width. Specify 0 to avoid truncation entirely.
- -a, --at=TREEOBJ
Perform the directory lookup relative to a directory reference in RFC 11 tree object format.
ls
Display directory referred to by key, or "." (root) if unspecified. This sub-command is intended to mimic ls(1) behavior in a limited way.
- -R, --recursive
List directory recursively.
- -d, --directory
List directory instead of contents.
- -w, --width=N
Limit output width to N columns. Specify 0 for unlimited output width.
- -1, --1
Force one entry per line.
- -F, --classify
Append key type indicator to key:
.for directory.@for symbolic link.
unlink
Remove key from the KVS.
- -O, --treeobj-root
After the commit has completed, display the new root directory reference in RFC 11 tree object format.
- -b, --blobref
After the commit has completed, display the new root directory reference as a single blobref.
- -s, --sequence
After the commit has completed, display the new root sequence number or "version".
- -R, --recursive
Specify recursive removal of a directory.
- -f, --force
Ignore nonexistent keys.
link
Create a new name for target, similar to a symbolic link. target does not have to exist. If linkname exists, it is overwritten.
- -T, --target-namespace=NAME
Specify an alternate namespace for target. By default, target is in the same namespace as linkname.
- -O, --treeobj-root
After the commit has completed, display the new root directory reference in RFC 11 tree object format.
- -b, --blobref
After the commit has completed, display the new root directory reference as a single blobref.
- -s, --sequence
After the commit has completed, display the new root sequence number or "version".
readlink
Print the symbolic link target of key. The target may be another key name,
or a namespace::key tuple. It is an error if key is not a
symbolic link.
- -a, --at=TREEOBJ
Perform the lookup relative to a directory reference in RFC 11 tree object format.
- -o, --namespace-only
Print only the namespace name if the target has a namespace prefix.
- -k, --key-only
Print only the key if the target has a namespace prefix.
mkdir
Create an empty directory. If key exists, it is overwritten.
- -O, --treeobj-root
After the commit has completed, display the new root directory reference in RFC 11 tree object format.
- -b, --blobref
After the commit has completed, display the new root directory reference as a single blobref.
- -s, --sequence
After the commit has completed, display the new root sequence number or "version".
dropcache
Tell the local KVS to drop any cache it is holding.
- -a, --all
Publish an event across the Flux instance instructing the KVS module on all ranks to drop their caches.
copy
Copy source key to destination key. If a directory is copied, a new reference is created.
- -S, --src-namespace=NAME
Specify the source namespace. By default, the primary KVS namespace or the value of
FLUX_KVS_NAMESPACEis used.
- -D, --dst-namespace=NAME
Specify the destination namespace. By default, the primary KVS namespace or the value of
FLUX_KVS_NAMESPACEis used.
move
Copy source key to destination key and unlink source.
- -S, --src-namespace=NAME
Specify the source namespace. By default, the primary KVS namespace or the value of
FLUX_KVS_NAMESPACEis used.
- -D, --dst-namespace=NAME
Specify the destination namespace. By default, the primary KVS namespace or the value of
FLUX_KVS_NAMESPACEis used.
getroot
Retrieve the current KVS root directory reference, displaying it as an RFC 11 dirref object unless otherwise specified.
- -o, --owner
Display the numerical user ID that owns the target namespace.
- -b, --blobref
Display the root directory reference as a single blobref.
- -s, --sequence
Display the root sequence number or "version".
version
Display the current KVS version, an integer value. The version starts at zero and is incremented on each KVS commit. Note that some commits may be aggregated for performance and the version will be incremented once for the aggregation, so it cannot be used as a direct count of commit requests.
wait
Block until the KVS version reaches version or greater. A simple form of synchronization between peers is: node A puts a value, commits it, reads version, sends version to node B. Node B waits for version, gets value.
namespace create
Create a new KVS namespace.
- -o, --owner=UID
Set the owner of the namespace to UID. If unspecified, the owner is set to the Flux instance owner.
- -r, --rootref=TREEOBJ
Initialize namespace with specific root directory reference If unspecified, an empty directory is referenced.
namespace remove
Remove a KVS namespace. Removal is not guaranteed to be complete when the command returns.
namespace list
List all current namespaces, with owner and flags.
eventlog get
Display the contents of an RFC 18 KVS eventlog referred to by key.
- -W, --waitcreate
If the key does not exist, wait until it does.
- -w, --watch
Monitor the eventlog, displaying new events as they are appended.
- -u, --unformatted
Display the eventlog in raw RFC 18 form.
- -H, --human
Display the eventlog in human-readable form.
- -L, --color[=WHEN]
Control output colorization. The optional argument WHEN can be one of 'auto', 'never', or 'always'. The default value of WHEN if omitted is 'always'. The default is 'auto' if the option is unused.
eventlog append
Append an event to an RFC 18 KVS eventlog referred to by key. The event name and optional context are specified on the command line.
- -t, --timestamp=SEC
Specify timestamp in decimal seconds since the UNIX epoch (UTC), otherwise the current wall clock is used.
eventlog wait-event
Wait for a specific event to occur in an RFC 18 KVS eventlog referred to by key.
- -W, --waitcreate
If the key does not exist, wait until it does.
- -t, --timeout=SEC
Timeout after SEC if the specified event has not occurred by then.
- -u, --unformatted
Display the event(s) in raw RFC 18 form.
- -q, --quiet
Do not display the matched event.
- -v, --verbose
Display events prior to the matched event.
- -H, --human
Display eventlog events in human-readable form.
- -L, --color[=WHEN]
Control output colorization. The optional argument WHEN can be one of 'auto', 'never', or 'always'. The default value of WHEN if omitted is 'always'. The default is 'auto' if the option is unused.
RESOURCES
Flux: http://flux-framework.org
Flux RFC: https://flux-framework.readthedocs.io/projects/flux-rfc