flux.constraint.parser module

class flux.constraint.parser.ConstraintLexer(**kw_args)

Bases: object

Simple constraint query syntax lexical analyzer based on RFC 35. Used mainly as the lexer for BaseConstraintParser

input(data)

states = (('squoting', 'exclusive'), ('dquoting', 'exclusive'))

t_AND(t)

&{1,2}|andb

t_ANY_error(t)

t_LPAREN(t)

(

t_NEGATE(t)

t_NOT(t)

notb

t_OR(t)

|{1,2}|orb

t_QUOTE(t)

'|"

t_RPAREN(t)

)

t_TOKEN(t)

[^()|&s"']+

t_dquoting_QUOTE(t)

"

t_dquoting_TOKEN(t)

([^"])+

t_dquoting_eof(t)

t_dquoting_ignore = ''

t_eof(t)

t_ignore = ' \t\r\n\x0c\x0b'

t_squoting_QUOTE(t)

'

t_squoting_TOKEN(t)

([^'])+

t_squoting_eof(t)

t_squoting_ignore = ''

tokens = ('NOT', 'AND', 'OR', 'NEGATE', 'LPAREN', 'RPAREN', 'TOKEN', 'QUOTE')

class flux.constraint.parser.ConstraintParser(lexer=None, optimize=True, debug=False, write_tables=False, **kw_args)

Bases: object

Base constraint query syntax parser class.

This class implements an RFC 35 compliant constraint query syntax parser with the following simplified grammar:

expr : expr expr
 | expr and expr
 | expr or expr
 | not expr
 | '(' expr ')'
 | '-' term
 | term

and : &{1,2}|and|AND

or : \|{1,2}|or|OR

not : not|NOT

term : \w*:?.+  # i.e. [operator:]operand

Where a term is a constraint operation which has the form '[operator:]operand'. If the token does not include a :, then a class default operator may optionally be substituted, e.g. "operand" becomes "default:operand".

By default, operand is included as a single entry in the RFC 31 values array for the operator, i.e. {"operator":["operand"]}. However, if operator appears in the self.split_values dict of the parser object, then the corresponding string will be used to split value into multiple entries. E.g. if

split_values = { "op": "," }

Then op:foo,bar would result in {"op":["foo","bar"]}.

Terms are joined by AND unless OR is specified, such that a b c is the same as a && b && c. A term can be negated with - (e.g. -a b c is equivalent to (not a)&b&c), but to negate a whole expression, NOT must be used (e.g. -(a|b) is a syntax error, use not (a|b) instead).

As a result of parsing, an RFC 31 constraint object is returned as a Python dict.

operator_map

A mapping of operator names, used to specify default and shorthand operator names. To configura a default operator, specify None as a key, e.g.

operator_map = { None: "name" }

Type

dict

split_values

A mapping of operator name to string on which to split values of that operator. For instance {"op": ","} would autosplit operator op values on comma.

Type

dict

combined_terms

A set of operator terms whose values can be combined when joined with the AND logical operator. E.g. if "test" is in combined_terms, then

test:a test:b

would produce

{"test": ["a", "b"]}

instead of

{"and": [{"test": ["a"]}, {"test": ["b"]}]}

Type

set

Subclasses can set the values of the above attributes to create a custom parser with default operators, split value handling, and set of combined terms.

E.g.:

class MyConstraintParser(ConstraintParser):
    operator_map = { None: "default", "foo": "long_foo" }
    split_values = { "default": "," }
    combined_terms = set("default")

By default there is no mapping for None which means each term requires an operator.

combine_like_terms(p1, p2)

combined_terms = {}

static invalid_operator(op)

operator_map = {}

p_error(p)

p_expression_and(p)

expression : expression AND expression

p_expression_or(p)

expression : expression OR expression

p_expression_parens(p)

expression : LPAREN expression RPAREN

p_expression_space(p)

expression : expression expression %prec AND

p_expression_token(p)

token : TOKEN

p_expression_unot(p)

expressionNOT expression

NEGATE token

p_quoted_token(p)

token : QUOTE TOKEN QUOTE

p_token(p)

expression : token

parse(query, **kw_args)

precedence = (('left', 'OR'), ('left', 'AND'), ('right', 'NOT'), ('right', 'NEGATE'))

split_values = {}

exception flux.constraint.parser.ConstraintSyntaxError

Bases: Exception

Specialized SyntaxError exception to allow ConstraintParser to throw a SyntaxError without PLY trying to force recovery.