flux.shape.parser module

class flux.shape.parser.ShapeLexer(**kw_args)

Bases: object

Simple job shape jobspec resources syntax lexical analyzer based on RFC 46. Used mainly as the lexer for BaseShapeParser

end_quote_helper(t)

input(data)

literals = ',;:/+-'

states = (('squoting', 'exclusive'), ('dquoting', 'exclusive'), ('count', 'exclusive'), ('dictkey', 'exclusive'), ('dictvalue', 'exclusive'))

t_ANY_LBRACE(t): {

t_ANY_LBRACKET(t): [

t_ANY_QUOTE(t): "|'

t_ANY_RBRACE(t): }

t_ANY_RBRACKET(t): ]

t_ANY_error(t)

t_EQUALS(t): =

t_TYPE(t): [^={}[]/;s"']+

t_count_IDSET(t): [1-9]d*((?=,)|-[1-9]d*(?=,))(,[1-9]d*(-[1-9]d*)?)*

t_count_INTEGER(t): [1-9]d*

t_count_OPERATOR(t): :[+*^]

t_count_end(t): (?=[{/;])

t_count_ignore = ' \t\r\n\x0c\x0b'

t_dictkey_COLON(t): :

t_dictkey_KEY(t): (?![+-])[^:,{}]+

t_dictkey_ignore = ' \t\r\n\x0c\x0b'

t_dictvalue_ARRAY(t): [.*]

t_dictvalue_COMMA(t): ,

t_dictvalue_NUMBER(t): -?(0|[1-9]d*)(.d*)?((e|E)(-|+)?d+)?(?=[},])

t_dictvalue_RBRACE(t): }

t_dictvalue_STRING(t): [^,{}[]]+

t_dictvalue_ignore = ' \t\r\n\x0c\x0b'

t_dquoting_QSTRING(t): [^"]+

t_dquoting_QUOTE(t): "

t_dquoting_ignore = ''

t_ignore = ' \t\r\n\x0c\x0b'

t_squoting_QSTRING(t): [^']+

t_squoting_QUOTE(t): '

t_squoting_dquoting_eof(t)

t_squoting_ignore = ''

tokens = ('TYPE', 'SLOT', 'OPERATOR', 'IDSET', 'INTEGER', 'KEY', 'FALSE', 'NULL', 'TRUE', 'NUMBER', 'STRING', 'QSTRING', 'ARRAY', 'LBRACKET', 'RBRACKET', 'LBRACE', 'RBRACE')

class flux.shape.parser.ShapeParser(lexer=None, optimize=True, debug=False, write_tables=False, **kw_args)

Bases: object

Base RFC 46 job shape resources syntax parser class.

This class implements an RFC 46 compliant Jobspec resources syntax parser with the following simplified grammar:

resources   : list

list        : '[' vertex (';' vertex)* ']'
            | vertex

vertex      : 'slot' ('=' count)? ('{' label (',' item)* '}')? '/' list
            | type ('=' count)? dict? ('/' list)?

type        : STRING

label       : STRING

count       : '[' count-value ']'
            | count-value

count-value : RANGE
            | IDSET
            | INTEGER

dict        : '{' (item (',' item)*)? '}'

item        : key ':' VALUE
            | (+|-)? key

key         : 'x'
            | STRING

The result of parsing is an RFC 14 resources list returned as a Python list.

p_count(p): count : count_value

p_count_brackets(p): count : LBRACKET count_value RBRACKET

p_count_value(p)

count_valueIDSET: INTEGER

p_count_value_range(p)

count_valueINTEGER '-' INTEGER: INTEGER '-' INTEGER ':' INTEGER

INTEGER '-' INTEGER ':' INTEGER OPERATOR

p_count_value_range_nomax(p)

count_valueINTEGER '+': INTEGER '+' ':' INTEGER

INTEGER '+' ':' INTEGER OPERATOR

p_dict(p): dict : LBRACE items RBRACE

p_dict_empty(p): dict : LBRACE RBRACE

p_error(p)

p_item_key(p): item : key

p_item_key_value(p): item : key ':' value

p_item_minus(p): item : '-' key

p_item_plus(p): item : '+' key

p_items_multi(p): items : item ',' items

p_items_single(p): items : item

p_key(p)

keyKEY: QSTRING

p_list_brackets(p): list : LBRACKET vertices RBRACKET

p_list_single(p): list : vertex

p_non_slot(p)

non_slotTYPE: QSTRING

p_non_slot_count(p)

non_slotTYPE count: QSTRING count

p_non_slot_count_dict(p)

non_slotTYPE count dict: QSTRING count dict

p_non_slot_dict(p)

non_slotTYPE dict: QSTRING dict

p_resources(p): resources : list

p_slot(p): slot : slot_tc

p_slot_label(p): slot : slot_tc LBRACE key RBRACE

p_slot_label_dict(p): slot : slot_tc LBRACE key ',' items RBRACE

p_slot_tc(p): slot_tc : SLOT

p_slot_tc_count(p): slot_tc : SLOT count

p_value(p)

valueFALSE: NULL

TRUE

dict

ARRAY

NUMBER

STRING

QSTRING

p_vertex(p): vertex : non_slot

p_vertex_with(p)

vertexnon_slot '/' list: slot '/' list

p_vertices_multi(p): vertices : vertex ';' vertices

p_vertices_single(p): vertices : vertex

parse(query, **kw_args)

exception flux.shape.parser.ShapeSyntaxError

Bases: Exception

Specialized SyntaxError exception to allow ShapeParser to throw a SyntaxError without PLY trying to force recovery.