PySlurm: Slurm Interface to python

Authors:Mark Roberts, Giovanni Torres
Date:Feb 03, 2019
Version:17.11.0.15

This module provides a low-level Python wrapper around the Slurm C-API using Cython.

Contents

Block Class

class pyslurm.block

Class to access/update slurm block Information.

find(self, name='', val='')

Search for property and associated value in the block data.

Parameters:
  • name (str) – key string to search
  • value – value to match
Returns:

List of IDs that match

Return type:

list

find_id(self, blockID=None)

Retrieve block ID data.

Parameters:blockID (str) – Block key string to search
Returns:Dictionary of values for given block key
Return type:dict
get(self)

Get slurm block information.

Returns:Dictionary whose key is the Block ID
Return type:dict
ids(self)

Return the block IDs from retrieved data.

Returns:Dictionary of block IDs
Return type:dict
lastUpdate(self)

Get the time (epoch seconds) the retrieved data was updated.

Returns:epoch seconds
Return type:integer
load(self)

Load slurm block information.

print_info_msg(self, int oneLiner=0)

Output information about all slurm blocks

This is based upon data returned by the slurm_load_block.

Parameters:oneLiner (int) – Print information on one line - 0 (Default), 1
update(self, blockID, int blockOP=0)
update_error(self, blockID)

Set slurm block to ERROR state.

Parameters:blockID (string) – The ID string of the block
update_free(self, blockID)

Set slurm block to FREE state.

Parameters:blockID (string) – The ID string of the block
update_recreate(self, blockID)

Set slurm block to RECREATE state.

Parameters:blockID (string) – The ID string of the block
update_remove(self, blockID)

Set slurm block to REMOVE state.

Parameters:blockID (string) – The ID string of the block
update_resume(self, blockID)

Set slurm block to RESUME state.

Parameters:blockID (string) – The ID string of the block

Config Class

class pyslurm.config

Class to access slurm config Information.

display_all(self)

Print slurm control configuration information.

find_id(self, char *keyID='')

Retrieve config ID data.

Parameters:keyID (str) – Config key string to search
Returns:Dictionary of values for given config key
Return type:dict
get(self)

Return the slurm control configuration information.

Returns:Configuration data
Return type:dict
ids(self)

Return the config IDs from retrieved data.

Returns:Dictionary of config key IDs
Return type:dict
key_pairs(self)

Return a dict of the slurm control data as key pairs.

Returns:Dictionary of slurm key-pair values
Return type:dict
lastUpdate(self)

Get the time (epoch seconds) the retrieved data was updated.

Returns:epoch seconds
Return type:integer

FrontEnd Class

class pyslurm.front_end

Class to access/update slurm front end node information.

get(self)

Get front end node information.

Returns:Dictionary whose key is the Topology ID
Return type:dict
ids(self)

Return the node IDs from retrieved data.

Returns:Dictionary of node IDs
Return type:dict
lastUpdate(self)

Return last time (sepoch seconds) the node data was updated.

Returns:epoch seconds
Return type:integer
load(self)

Load slurm front end node information.

HostList Class

class pyslurm.hostlist

Wrapper class for Slurm hostlist functions.

count(self)
create(self, hostnames=None)
destroy(self)
find(self, hostname)
get(self)
get_list(self)

Get the list of hostnames composing the hostlist.

For example with a hostlist created with “tux[1-3]” -> [ ‘tux1’, tux2’, ‘tux3’ ].

Returns:the list of hostnames in case of success or None on error.
Return type:list
pop(self)
push(self, hosts)
push_host(self, host)
ranged_string(self)
shift(self)
uniq(self)

Job Class

class pyslurm.job

Class to access/modify Slurm Job Information.

find(self, name='', val='')

Search for a property and associated value in the retrieved job data.

Parameters:
  • name (str) – key string to search
  • value (str) – value string to match
Returns:

List of IDs that match

Return type:

list

find_id(self, jobid)

Retrieve job ID data.

This method accepts both string and integer formats of the jobid. It calls slurm_xlate_job_id() to convert the jobid appropriately. This works for single jobs and job arrays.

Parameters:jobid (str) – Job id key string to search
Returns:List of dictionary of values for given job id
Return type:list
find_user(self, user)

Retrieve a user’s job data.

This method calls slurm_load_job_user to get all job_table records associated with a specific user.

Parameters:user (str) – User string to search
Returns:Dictionary of values for all user’s jobs
Return type:dict
get(self)

Get all slurm jobs information.

This method calls slurm_load_jobs to get job_table records for all jobs

Returns:Data where key is the job name, each entry contains a dictionary of job attributes
Return type:dict
ids(self)

Return the job IDs from retrieved data.

Returns:Dictionary of job IDs
Return type:dict
lastUpdate(self)

Get the time (epoch seconds) the job data was updated.

Returns:epoch seconds
Return type:integer
print_job_info_msg(self, int oneLiner=0)

Print the data structure describing all job step records.

The job step records are loaded by the slurm_get_job_steps function.

Parameters:Flag (int) – Default=0
slurm_job_batch_script(self, jobid)

Retrieve the batch script for a given jobid.

Parameters:jobid (str) – Job id key string to search
Returns:String output of a jobid’s batch script
Return type:str
submit_batch_job(self, job_opts)

Submit batch job. * make sure options match sbatch command line opts and not struct member names.

JobStep Class

class pyslurm.jobstep

Class to access/modify Slurm Jobstep Information.

find(self, jobID=-1, stepID=-1)
get(self)

Get slurm jobstep information.

Returns:Data whose key is the jobstep ID.
Return type:dict
ids(self)
lastUpdate(self)

Get the time (epoch seconds) the jobstep data was updated.

Returns:epoch seconds
Return type:integer
layout(self, uint32_t JobID=0, uint32_t StepID=0)

Get the slurm job step layout from a given job and step id.

Parameters:
  • JobID (int) – slurm job id (Default=0)
  • StepID (int) – slurm step id (Default=0)
Returns:

List of job step layout.

Return type:

list

Node Class

class pyslurm.node

Class to access/modify/update Slurm Node Information.

find_id(self, nodeID)

Get node information for a given node.

Parameters:nodeID (str) – Node key string to search
Returns:Dictionary of values for given node
Return type:dict
get(self)

Get all slurm node information.

Returns:Dictionary of dictionaries whose key is the node name.
Return type:dict
get_node(self, nodeID)

Get single slurm node information.

Parameters:nodeID (str) – Node key string to search. Default NULL.
Returns:Dictionary of give node info data.
Return type:dict
ids(self)

Return the node IDs from retrieved data.

Returns:Dictionary of node IDs
Return type:dict
lastUpdate(self)

Return last time (epoch seconds) the node data was updated.

Returns:epoch seconds
Return type:integer
parse_gres(self, gres_str)
print_node_info_msg(self, int oneLiner=False)

Output information about all slurm nodes.

Parameters:oneLiner (int) – Print on one line - False (Default) or True
update(self, dict node_dict)

Update slurm node information.

Parameters:node_dict (dict) – A populated node dictionary, an empty one is created by create_node_dict
Returns:0 for success or -1 for error, and the slurm error code is set appropriately.
Return type:integer

Partition Class

class pyslurm.partition

Class to access/modify Slurm Partition Information.

create(self, dict Partition_dict)

Create a slurm partition.

Parameters:partition_dict (dict) – A populated partition dictionary, an empty one can be created by create_partition_dict
Returns:0 for success or -1 for error, and the slurm error code is set appropriately.
Return type:integer
delete(self, PartID)

Delete a give slurm partition.

Parameters:PartID (string) – Name of slurm partition
Returns:0 for success else set the slurm error code as appropriately.
Return type:integer
find(self, name='', val='')

Search for a property and associated value in the retrieved partition data.

Parameters:
  • name (str) – key string to search
  • value (str) – value string to match
Returns:

List of IDs that match

Return type:

list

find_id(self, partID)

Get partition information for a given partition.

Parameters:partID (str) – Partition key string to search
Returns:Dictionary of values for given partition
Return type:dict
get(self)

Get all slurm partition information

Returns:Dictionary of dictionaries whose key is the partition name.
Return type:dict
ids(self)

Return the partition IDs from retrieved data.

Returns:Dictionary of partition IDs
Return type:dict
lastUpdate(self)

Return time (epoch seconds) the partition data was updated.

Returns:epoch seconds
Return type:integer
print_info_msg(self, int oneLiner=0)

Display the partition information from previous load partition method.

Parameters:oneLiner (int) – Display on one line (default=0)
update(self, dict Partition_dict)

Update a slurm partition.

Parameters:partition_dict (dict) – A populated partition dictionary, an empty one is created by create_partition_dict
Returns:0 for success, -1 for error, and the slurm error code is set appropriately.
Return type:integer

Reservation Class

class pyslurm.reservation

Class to access/update/delete slurm reservation Information.

create(self, dict reservation_dict={})

Create slurm reservation.

delete(self, ResID)

Delete slurm reservation.

Returns:0 for success or a slurm error code
Return type:integer
find(self, name='', val='')

Search for property and associated value in reservation data.

Parameters:
  • name (str) – key string to search
  • value (str) – value string to match
Returns:

List of IDs that match

Return type:

list

find_id(self, resID)

Retrieve reservation ID data.

Parameters:resID (str) – Reservation key string to search
Returns:Dictionary of values for given reservation key
Return type:dict
get(self)

Get slurm reservation information.

Returns:Data whose key is the Reservation ID
Return type:dict
ids(self)

Return a list of reservation IDs from retrieved data.

Returns:Dictionary of reservation IDs
Return type:dict
lastUpdate(self)

Get the time (epoch seconds) the reservation data was updated.

Returns:epoch seconds
Return type:integer
load(self)
print_reservation_info_msg(self, int oneLiner=0)

Output information about all slurm reservations.

Parameters:Flags (int) – Print on one line - 0 (Default) or 1
update(self, dict reservation_dict={})

Update a slurm reservation attributes.

Returns:0 for success or -1 for error, and the slurm error code is set appropriately.
Return type:integer

Slurmdb Events Class

class pyslurm.slurmdb_events

Class to access Slurmdbd events information.

get(self)

Get slurm events information.

Returns:Dictionary whose keys are the events ids
Return type:dict
set_event_condition(self, start_time, end_time)

Limit the next get() call to conditions that existed after and before a certain time.

Parameters:
  • start_time – Select conditions that existed after this timestamp
  • end_time – Select conditions that existed before this timestamp

Slurmdb Reservations Class

class pyslurm.slurmdb_reservations

Class to access Slurmdbd reservations information.

get(self)

Get slurm reservations information.

Returns:Dictionary whose keys are the reservations ids
Return type:dict
set_reservation_condition(self, start_time, end_time)

Limit the next get() call to reservations that start after and before a certain time.

Parameters:
  • start_time – Select reservations that start after this timestamp
  • end_time – Select reservations that end before this timestamp

Slurmdb Clusters Class

class pyslurm.slurmdb_clusters

Class to access Slurmdbd Clusters information.

get(self)

Get slurm clusters information.

Returns:Dictionary whose keys are the clusters ids
Return type:dict
set_cluster_condition(self, start_time, end_time)

Limit the next get() call to clusters that existed after and before a certain time.

Parameters:
  • start_time – Select clusters that existed after this timestamp
  • end_time – Select clusters that existed before this timestamp

Slurmdb Jobs Class

class pyslurm.slurmdb_jobs

Class to access Slurmdbd Jobs information.

get(self, jobids=[], starttime=0, endtime=0)

Get Slurmdb information about some jobs.

Input formats for start and end times: * today or tomorrow * midnight, noon, teatime (4PM) * HH:MM [AM|PM] * MMDDYY or MM/DD/YY or MM.DD.YY * YYYY-MM-DD[THH[:MM[:SS]]] * now + count [minutes | hours | days | weeks] * Invalid time input results in message to stderr and return value of zero.

Parameters:
  • jobids – Ids of the jobs to search. Defaults to all jobs.
  • starttime – Select jobs eligible after this timestamp
  • endtime – Select jobs eligible before this timestamp
Returns:

Dictionary whose key is the JOBS ID

Return type:

dict

Statistics Class

class pyslurm.statistics
get(self) → dict

Get slurm statistics information.

Return type:dict
reset(self) → int

Reset scheduling statistics

This method required root privileges.

Topology Class

class pyslurm.topology

Class to access/update slurm topology information.

display(self)

Display topology information to standard output.

get(self)

Get slurm topology information.

Returns:Dictionary whose key is the Topology ID
Return type:dict
lastUpdate(self)

Get the time (epoch seconds) the retrieved data was updated.

Returns:epoch seconds
Return type:integer
load(self)

Load slurm topology information.

Trigger Class

class pyslurm.trigger
clear(self, TriggerID=0, UserID=slurm.NO_VAL, ID=0)

Clear or remove a slurm trigger.

Parameters:
  • TriggerID (string) – Trigger Identifier
  • UserID (string) – User Identifier
  • ID (string) – Job Identifier
Returns:

0 for success or a slurm error code

Return type:

integer

get(self)

Get the information on slurm triggers.

Returns:Where key is the trigger ID
Return type:dict
set(self, dict trigger_dict)

Set or create a slurm trigger.

Parameters:trigger_dict (dict) – A populated dictionary of trigger information
Returns:0 for success or -1 for error, and the slurm error code is set appropriately.
Return type:integer

Indices and tables