Flux Workflow Examples

The examples contained here demonstrate and explain some simple use-cases with Flux, and make use of Flux’s command-line interface (CLI), Flux’s C library, and the Python and Lua bindings to the C library. The entire set of examples can be downloaded by cloning the Github repo.

The examples assume that you have installed:

  1. A recent version of Flux

  2. Python 3.6+

  3. Lua 5.1+

CLI: Job Submission

Launch a flux instance and schedule/launch compute and io-forwarding jobs on separate nodes using the CLI

Python: Job Submission

Schedule/launch compute and io-forwarding jobs on separate nodes using the Python bindings

Python: Job Submit/Wait

Submit jobs and wait for them to complete using the Flux Python bindings

Python: Asynchronous Bulk Job Submission

Asynchronously submit jobspec files from a directory and wait for them to complete in any order

Python: Tracking Job Status and Events

Submit job bundles and wait until all jobs complete

Python: Job Cancellation

Cancel a running job

Lua: Use Events

Use events to synchronize compute and io-forwarding jobs running on separate nodes

Python: Simple KVS Example

Use KVS Python interfaces to store user data into KVS

CLI/Lua: Job Ensemble Submitted with a New Flux Instance

Submit job bundles, print live job events, and exit when all jobs are complete

CLI: Hierarchical Launching

Launch a large number of sleep 0 jobs

C/Lua: Use a Flux Comms Module

Use a Flux Comms Module to communicate with job elements

C/Python: A Data Conduit Strategy

Attach to a job that receives OS time data from compute jobs