flux.resource.Rv1Pool module

Rv1 resource pool: combined resource-set and scheduler-layer implementation.

Rv1Pool inherits from both Rv1Set (resource-set tracking) and ResourcePoolImplementation (scheduler-layer protocol). It adds per-rank availability and allocation tracking to the base Rv1Set representation.

The internal _ranks dict is extended with three extra keys beyond what Rv1Set uses:

{"hostname": str, "cores": frozenset, "gpus": frozenset,
 "up": bool, "allocated_cores": set, "allocated_gpus": set}

Rv1Pool supports GPU scheduling and worst-fit allocation. Constraints (properties, hostlist, ranks, boolean operators) are supported via the inherited _matches_constraint() method.

To obtain an Rv1Pool from an R JSON string or dict, call:

from flux.resource import ResourcePool
pool = ResourcePool(R)          # factory dispatch → Rv1Pool

or directly:

from flux.resource.Rv1Pool import Rv1Pool
pool = Rv1Pool(R)

class flux.resource.Rv1Pool.ResourceRequest(node_count, slot_count, slot_size, gpu_per_slot, duration, constraint, exclusive, jobspec=None)

Bases: object

Parsed resource request extracted from a jobspec.

Returned by Rv1Pool.parse_resource_request() and stored on PendingJob. Passed to Rv1Pool.alloc() so that the pool does not need to re-parse jobspec on every scheduling pass. Node and slot counts carry RFC 14 ranges (min/max).

node_count

RFC 14 node count (scaled by nodefactor), or None for slot-only layouts. For exclusive slot-only jobspecs this holds the slot count reinterpreted as node count.

Type: ResourceCount | None

slot_count

RFC 14 slot count. For node-based layouts this is the per-node slot count; for slot-only layouts it is the total slot count; for exclusive slot-only layouts it is ResourceCount(1, 1).

Type: ResourceCount

slot_size

Cores per slot.

Type: int

gpu_per_slot

GPUs per slot.

Type: int

duration

Walltime in seconds; 0.0 means unlimited.

Type: float

constraint: RFC 31 constraint expression (dict) or None.

exclusive

Whole-node exclusive allocation.

Type: bool

jobspec

The raw jobspec dict from which this request was parsed. Available for reading site-specific hints from attributes.system that the standard parser does not extract, or for forwarding the full jobspec to an external resource service.

Type: dict

nnodes

Minimum node count; derived from node_count.

Type: int

nnodes_max

Maximum node count; None for unbounded.

Type: int | None

nslots

Minimum total slot count; derived from slot_count and node_count.

Type: int

nslots_max

Maximum total slot count.

Type: int | None

constraint

duration

exclusive

classmethod from_jobspec(jobspec)

Parse a jobspec dict and return a ResourceRequest.

Walks the resource graph recursively, like libjjc, to support both RFC 25 V1 jobspecs and jobspecs with non-V1 resource hierarchies. Unknown resource types are skipped but their with children are still traversed. The version key value is not checked (see #6682).

RFC 14 range counts in all forms (integer, dict, RFC 45 string, idset string) on the node or slot resource are fully supported: the scheduler allocates as many resources as available up to the maximum.

Raises

ValueError -- If the jobspec cannot be parsed.
KeyError -- If required fields are missing.

gpu_per_slot

jobspec

property ncores: Total cores requested (nslots × slot_size).

property ngpus: Total GPUs requested (nslots × gpu_per_slot).

property nnodes: Minimum node count; 0 for slot-only layouts.

property nnodes_max: Maximum node count; 0 for slot-only; None for unbounded.

node_count

property nslots: Minimum total slot count.

property nslots_max: Maximum total slot count; None for unbounded.

slot_count

slot_size

class flux.resource.Rv1Pool.Rv1Pool(R, log=None, **kwargs)

Bases: Rv1Set, ResourcePoolImplementation

Pure-Python Rv1 resource pool combining resource-set and scheduler layers.

MRO: Rv1Pool → Rv1Set → ResourceSetImplementation → ResourcePoolImplementation → ABC → object

The _ranks dict carries three extra pool-specific keys beyond the base Rv1Set representation:

up              – True if rank is schedulable
allocated_cores – set of core IDs currently allocated on this rank
allocated_gpus  – set of GPU IDs currently allocated on this rank

alloc(jobid: int, request) → Rv1Pool

Allocate resources for jobid matching request.

Parameters

jobid (int) -- The job ID; stored in _job_state.
request -- A ResourceRequest from parse_resource_request().

Returns

A new Rv1Pool containing the allocated resources.

Raises

InsufficientResources -- Resources temporarily insufficient.
InfeasibleRequest -- Request structurally infeasible.

check_feasibility(request) → None

Check whether a request is structurally satisfiable.

Tests against total pool capacity (ignoring current allocations and availability) so that transient conditions don't affect the result.

Parameters: request -- A ResourceRequest from parse_resource_request().
Raises: InfeasibleRequest -- If the request can never be satisfied by this pool's total capacity.

copy() → Rv1Pool: Return a full independent copy preserving allocation state.

copy_allocated() → Rv1Set

Return an Rv1Set containing only the allocated resources.

Properties are intersected with the allocated ranks so that queue/property information is preserved (e.g. for sched.resource-status).

copy_down() → Rv1Set: Return an Rv1Set containing only the down ranks.

free(jobid: int, R=None, final: bool = False) → None

Return a job's allocated resources to this pool.

The scheduler allocation protocol may partially release a job's resources by rank; each partial free updates both the pool and the job's tracked allocation in _job_state.

Parameters

jobid -- ID of the job whose resources are being freed.
R -- Resource set to free. For a partial free this contains only the subset being freed; when final is True and R is provided it must exactly match the tracked allocation. If None, all tracked resources for the job are freed.
final -- If True, all resources remaining in the tracked allocation are freed and the job state is removed. For a partial free, R must be a subset of the tracked allocation. These checks catch job manager accounting errors.

Raises

ValueError -- If R contains ranks not in the job's tracked allocation, or if final is True and R does not match the tracked allocation exactly.

job_end_times() → List[Tuple[int, float]]: Return a list of (jobid, end_time) pairs for all tracked jobs.

known_options: frozenset = frozenset({}): Pool options recognised by this implementation. The scheduler logs a warning for any key in pool_kwargs not listed here.

mark_down(ids: str) → None: Mark ranks as not schedulable.

mark_up(ids: str) → None: Mark ranks as schedulable.

parse_resource_request(jobspec: dict) → ResourceRequest: Parse a V1 jobspec and return a ResourceRequest.

register_alloc(jobid: int, R: Rv1Pool) → None

Register an existing allocation during scheduler reconnect.

Raises: ValueError -- If R contains ranks absent from this pool, which indicates the job's resources are no longer available (e.g. the rank was excluded from configuration before the scheduler was reloaded). The caller should raise a fatal exception on the job.

remove_ranks(ranks) → None: Remove ranks from the pool (shrink event).

to_set() → Rv1Set

Return a topology+availability snapshot as an Rv1Set.

The returned set contains all ranks with their topology (cores, gpus) and current up/down state, but no allocation information.

update_expiration(jobid: int, expiration: float) → None: Update the end time for a tracked job.

version = 1