flux.job.Jobspec module

class flux.job.Jobspec.Jobspec(resources, tasks, **kwargs)

Bases: object

attributes
cwd

Get working directory of job.

dumps(**kwargs)
duration
environment

Get (entire) environment of job.

classmethod from_yaml_file(filename)
classmethod from_yaml_stream(yaml_stream)
resource_counts()

Compute the counts of each resource type in the jobspec

The following jobspec would return { "slot": 12, "core": 18, "memory": 242 }

- type: slot
  count: 2
  with:
    - type: core
      count: 4
    - type: memory
      count: 1
      unit: GB
- type: slot
  count: 10
  with:
    - type: core
      count: 1
    - type: memory
      count: 24
      unit: GB

Note

the current implementation ignores the unit label and assumes they are consist across resources

resource_walk()

Traverse the resources in the resources section of the jobspec.

Performs a depth-first, pre-order traversal. Yields a tuple containing (parent, resource, count). parent is None when resource is a top-level resource. count is the number of that resource including the multiplicative effects of the with clause in ancestor resources. For example, the following resource section, will yield a count of 2 for the slot and a count of 8 for the core resource:

- type: slot
  count: 2
  with:
    - type: core
      count: 4
resources
setattr(key, val)

set job attribute

setattr_shell_option(key, val)

set job attribute: shell option

stderr
stdin
stdout
tasks
top_level_keys = {'attributes', 'resources', 'tasks', 'version'}
version
class flux.job.Jobspec.JobspecV1(resources, tasks, **kwargs)

Bases: flux.job.Jobspec.Jobspec

classmethod from_batch_command(script, jobname, args=None, num_slots=1, cores_per_slot=1, gpus_per_slot=None, num_nodes=None, broker_opts=None)

Create a Jobspec describing a nested Flux instance controlled by a script.

The nested Flux instance will execute the script with the given command-line arguments after copying it and setting the executable bit. Conceptually, this differs from the from_nest_command, which also creates a nested Flux instance, in that it a) requires the initial program of the new instance to be an executable text file and b) creates the initial program from a string rather than using an executable existing somewhere on the filesystem.

Use setters to assign additional properties.

Parameters:
  • script -- contents of the script to execute, as a string. The script should have a shebang (e.g. #!/bin/sh) at the top.
  • jobname (str) -- name to use as the argv[0] for this job. This will be the default job name reported by Flux. (Note the actual argv is overridden by the job shell when executed.)
  • args (iterable of str) -- arguments to pass to script
  • num_slots -- number of resource slots to create. Slots are an abstraction, and are only used (along with cores_per_slot and gpus_per_slot) to determine the nested instance's allocation size and layout.
  • cores_per_slot -- number of cores to allocate per slot
  • gpus_per_slot -- number of GPUs to allocate per slot
  • num_nodes -- distribute allocated resource slots across N individual nodes
  • broker_opts (iterable of str) -- options to pass to the new Flux broker
classmethod from_command(command, num_tasks=1, cores_per_task=1, gpus_per_task=None, num_nodes=None)

Factory function that builds the minimum legal v1 jobspec.

Use setters to assign additional properties.

Parameters:
  • command (iterable of str) -- command to execute
  • num_tasks -- number of MPI tasks to create
  • cores_per_task -- number of cores to allocate per task
  • gpus_per_task -- number of GPUs to allocate per task
  • num_nodes -- distribute allocated tasks across N individual nodes
classmethod from_nest_command(command, num_slots=1, cores_per_slot=1, gpus_per_slot=None, num_nodes=None, broker_opts=None)

Create a Jobspec describing a nested Flux instance controlled by command.

Conceptually, this differs from the from_batch_command method in that a) the initial program of the nested Flux instance can be any executable on the file system, not just a text file and b) the executable is not copied at submission time.

Use setters to assign additional properties.

Parameters:
  • command (iterable of str) -- initial program for the nested Flux instance
  • num_slots -- number of resource slots to create. Slots are an abstraction, and are only used (along with cores_per_slot and gpus_per_slot) to determine the nested instance's allocation size and layout.
  • cores_per_slot -- number of cores to allocate per slot
  • gpus_per_slot -- number of GPUs to allocate per slot
  • num_nodes -- distribute allocated resource slots across N individual nodes
  • broker_opts (iterable of str) -- options to pass to the new Flux broker
flux.job.Jobspec.validate_jobspec(jobspec, require_version=None)

Validates the jobspec by attempting to construct a Jobspec object. If no exceptions are thrown during construction, then the jobspec is assumed to be valid and this function returns True. If the jobspec is invalid, the relevant exception is thrown (i.e., TypeError, ValueError, EnvironmentError)

By default, the validation function will read the version key in the jobspec to determine which Jobspec object to instantiate. An optional require_version is included to override this behavior and force a particular class to be used.

Parameters:
  • jobspec -- a Jobspec object or JSON string
  • require_version -- jobspec version to use, if not provided, the value of jobspec['version'] is used
Raises:
  • ValueError --
  • TypeError --
  • EnvironmentError --