flux.job.watcher module

class flux.job.watcher.JobProgressBar(flux_handle, starttime=None, jps=False, counter_width=3, update_interval=0.25)

Bases: ProgressBar

Progress bar for multiple jobs

The JobProgressBar class is a ProgressBar specific to monitoring progress of a group of jobs. It displays a progress bar with the number of pending, running, complete and failed jobs on the left, and a percent complete and elapsed timer by default on the right, with a progress bar in the middle.

Once a JobProgressBar object is initialized, jobs to track should be added with JobProgressBar.add_job or JobProgressBar.add_jobs. Subsequently, eventlog events for jobs are fed into the progress bar via JobProgressBar.process_event. To advance the progress bar, the update() method should be called.

jobs

list of JobStatus objects being monitored

starttime

start time used for the elapsed timer and jobs/s display. By default this will be the minimum submission time of all jobs being monitored. To choose a different starttime, manually set the starttime in the initializer.

total

total number of jobs being monitored

pending

current pending job count

running

current running job count

complete

current complete job count (successful jobs)

failed

current failed job count (failed, canceled, timeout)

add_job(job)

Begin monitoring the progress of a job.

Parameters

job (JobInfo) -- The job to begin monitoring

add_jobs(*jobs)

Add multiple jobs to a JobProgressBar instance

advance(**kwargs)

Advance progress bar (e.g. if one job has completed). :param kwargs: keyword args passed to ProgressBar.update

jobs_per_sec()

Return the current job throughput.

process_event(jobid, event=None)

Process an event for a job, updating job's progress if necessary.

Parameters

  • jobid -- job id

  • event -- event entry to process. If None, then the job is considered complete, i.e. no more events will be received for this job.

start()

Start JobProgressBar operation.

Initialize and start timer watchers, display initial progress bar, and if not set, initialize the elapsed start time.

update()

Update job state counts for ProgressBar and refresh display

class flux.job.watcher.JobStatus(job)

Bases: object

Simple convenience class for caching job "state" in the JobProgressBar and JobWatch classes.

id

The job id

status

This job's current simplified status (See below for possible status values)

exitcode

The exit code of the job (0 for success, otherwise failure)

Current valid statuses include: - pending - running - complete - failed

property active

True if the job is still active

add_event(name)

Add an event to the event counter

event_count(name)

Return the number of times event name has been seen for this job

has_event(name)

Return True if this job has seen event name

class flux.job.watcher.JobWatcher(flux_handle, jobs=None, progress=False, jps=False, wait='clean', watch=True, log_events=False, log_status=False, stdout=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, stderr=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, labelio=False, starttime=None)

Bases: object

Watch output and status for multiple jobs.

The JobWatcher class can watch the status, output, and logs for one or more jobs, optionally including a progress bar for use in a tty. This is the class that implements the --watch option of flux submit and flux bulksubmit.

class JobWatchStatus(job, stdout, stderr, wait='clean')

Bases: JobStatus

JobStatus class with extra attributes for use in JobWatcher

add_jobid(jobid, stdout=None, stderr=None, wait='clean')

Begin monitoring the progress of a job by jobid

Parameters

  • jobid (JobID) -- one or more jobs to monitor

  • stdout (TextIOWrapper) -- destination for stdout for this job or jobs. If None, use default for this JobWatcher instance.

  • stderr -- (TextIOWrapper): destination for stderr for this job or jobs (if None, then set to same location as stdout)

  • wait -- event at which to stop watching the job (default=clean)

add_jobs(*jobs, stdout=None, stderr=None, wait='clean')

Begin monitoring the progress of a set of jobs

Parameters

  • *jobs (JobID) -- one or more jobs to monitor

  • stdout (TextIOWrapper) -- destination for stdout for this job or jobs. If None, use default for this JobWatcher instance.

  • stderr -- (TextIOWrapper): destination for stderr for this job or jobs (if None, then set to same location as stdout)

  • wait -- event at which to stop watching the job (default=clean)

start()

Start JobWatcher progress bar if configured

stop()

Stop JobWatcher progress bar if configured