flux.testing.fake_resources module

Fake resource generation and installation for testing.

Provides a FakeResources ABC describing a synthetic resource set with a flat shape (nodes × cores × gpus), plus an InjectFakeResources implementation that encodes R via flux R encode and writes it to the resource.R KVS key. Intended for use by the fake-resources modprobe rc1 task (see flux-config-fake-resources(5)), which runs this code before the resource module loads so the synthetic R is in place at broker startup.

class flux.testing.fake_resources.FakeResources(nodes, cores_per_node=1, gpus_per_node=0, host_prefix='fake', amender=None)

Bases: ABC

Abstract base for synthetic resource sets with a flat shape.

A resource set described as a count of nodes, cores per node, and (optionally) GPUs per node. On-node topology (sockets, NUMA domains) is not modeled here; subclasses or callers needing topology should pass real hwloc XML, which flux R encode --local will consume, or override amend_R() to inject scheduler-specific metadata.

Subclasses must implement install(), which makes the resource set visible to the broker. Subclasses may also override amend_R() to mutate the encoded R before it is installed.

amend_R(R, hwloc_xml=None)

Hook to mutate R after generation, before KVS install.

Subclasses can override directly for complex amendment logic. Callers (typically the fake-resources modprobe rc1 task) can also pass amender= to the constructor to inject a callable taking (R, hwloc_xml=...) and returning amended R; the default amend_R() defers to it. Subclass overrides take precedence; a subclass that wants to incorporate self.amender should call super().amend_R(R, hwloc_xml=...).

Used to inject scheduler-specific metadata into R — for example, Fluxion JGF keys or property tags. When called against an XML-derived R, hwloc_xml is the loaded XML string; otherwise it is None.

abstract install()

Make these resources visible to the broker.

Concrete subclasses may extend this signature with additional arguments.

saturation_count(slot_cores=1, slot_gpus=0)

Number of slot-shape jobs needed to saturate this resource set.

See saturation_count() for the algorithm. This method is a thin wrapper that pulls nodes, cores_per_node, and gpus_per_node from self; it exists so callers holding a FakeResources object don't have to unpack its attributes. The function form is what flux schedbench calls for the real-resource path, where there's no FakeResources object — the shape comes from a flux.resource.resource_list query.

property total_cores

Total core count across all nodes.

property total_gpus

Total GPU count across all nodes.

class flux.testing.fake_resources.InjectFakeResources(nodes, cores_per_node=1, gpus_per_node=0, host_prefix='fake', hwloc_xml_path=None, verbose=False, log=None, amender=None)

Bases: FakeResources

FakeResources that writes synthetic R to the KVS.

install() encodes R from the configured shape (or from an hwloc XML file, if hwloc_xml_path is set) and writes it to resource.R. The caller is responsible for arranging the write to happen before the resource module reads its initial R, and for setting any resource-module options needed to accept synthetic R (noverify, monitor-force-up). The fake-resources modprobe rc1 task handles both.

If hwloc_xml_path is set, R is encoded by passing the XML file to flux R encode --xml=FILE rather than from the numeric cores_per_node / gpus_per_node fields. The XML's loaded contents are passed to amend_R() so subclasses can use them.

The log argument is a callable taking a single string (default _stderr_log()). Pass LOGGER.info to integrate with a CLI's logging configuration, or lambda _: None to suppress output entirely.

Writing an amender

An amender is a Python callable that mutates the encoded R before it is written to the KVS. Use it to inject scheduler-specific metadata that the bare flux R encode output doesn't carry, most notably the contents of the scheduling key, but may also include things like node properties or custom attributes for property-based schedulers.

An amender has the signature:

def amend(R, hwloc_xml=None): # mutate R in place, or build a new dict

R["scheduling"] = {...} return R

Where:

  • R is the parsed R dict (RFC 20) produced by flux R encode, ready to mutate. The amender owns it for the duration of the call; either mutate in place and return the same dict, or build a new one and return that.

  • hwloc_xml is the contents of the hwloc XML file when hwloc_xml_path was set on the constructor, or None otherwise. Useful for amenders that derive their additions from the topology (Fluxion's JGF, for example, walks the XML to build a graph).

  • The return value is what gets written to resource.R in KVS.

The amender is passed to the constructor as the amender argument and may be either:

  • A callable, ready to invoke. Use this when the amender is defined in the same source file or pulled from a Python package the caller already imported.

  • A string in one of two forms, resolved at construction time via load_amender():

    • "module.path:function_name" — importlib loads module.path and looks up function_name. Use this when the amender ships in a package on PYTHONPATH.

    • Anything without a colon is treated as a filesystem path; the file is loaded as a Python module and its amend (literally that name) attribute is used. Use this for ad-hoc test amenders or for passing an amender from a modprobe config without packaging it.

The string forms are what the [fake-resources] amend-r TOML key in flux-config-fake-resources(5) accepts; the rc1 task forwards the TOML value straight to this constructor.

Subclassing FakeResources and overriding amend_R() is an alternative to passing an amender — choose subclassing when the amendment logic is non-trivial (multiple methods, internal state) and the callable form when the logic fits in one function. Subclass overrides take precedence over the constructor amender; a subclass that wants to incorporate the constructor amender should call super().amend_R(R, hwloc_xml=...).

install(handle=None)

Encode R from the configured shape and install it into the resource.R KVS key.

Parameters

  • handle -- an open flux.Flux handle used for the KVS

  • None (put + commit. If) --

  • via (a fresh handle is opened) --

:param flux.Flux().:

flux.testing.fake_resources.build_R_encode_args(fr)

Build argv for flux R encode from a FakeResources.

Exposed separately from InjectFakeResources so unit tests can verify command construction without requiring a broker.

Idset-shaped arguments (-r, -c, -g) are formatted via flux.idset.IDset rather than f"0-{n-1}". A degenerate single-element range like "0-0" is rejected when flux R encode parses it as an idset, which silently produced R without the affected resource type (the gpus_per_node=1 case originally surfaced this). IDset emits "0" for a one-element set, which parses cleanly.

flux.testing.fake_resources.load_amender(spec)

Resolve a TOML amend-r spec to a callable.

Two forms are supported:

  • module.path:function_name — imports module.path and returns its function_name attribute. Useful when the amender ships with a Python package on PYTHONPATH (e.g. a Fluxion JGF helper).

  • Anything without a : is treated as a filesystem path; the file is loaded as a Python module and its amend callable is returned. Useful for ad-hoc test amenders that don't warrant a package.

The returned callable should have signature amend(R, hwloc_xml=None) -> R. It is invoked from FakeResources.amend_R() when set via the amender constructor argument.

Raises RuntimeError with a descriptive message on any failure (file not found, import error, missing attribute), so configuration mistakes surface clearly from the modprobe rc1 task rather than as a generic ImportError.

flux.testing.fake_resources.saturation_count(nodes, cores_per_node, gpus_per_node, slot_cores=1, slot_gpus=0)

Return the number of slot-shape jobs that saturate a flat resource set.

A slot is described by its core and GPU counts; this returns the number of such slots that fit across nodes machines of cores_per_node cores and gpus_per_node GPUs each, taking the more-constraining of cores and GPUs into account. Pure arithmetic — no broker access, no FakeResources object required — so it's reusable from code paths that gather the shape from flux.resource.resource_list() instead.

Either slot_cores or slot_gpus must be positive.

Non-uniformity note: this assumes uniform cores/GPUs per node. Real broker resources may not be uniform; callers that derive the shape from a broker query typically average across nodes, which can over- or under-estimate true capacity by one slot per heterogeneous node. The error is bounded and not a correctness issue for benchmarks that measure aggregate rates.