The query tokenizer will now correctly extract longer operators in all situations. For example if a "^" operator had higher precedence than a "^^" operator the tokenizer was previously parsing "^^" as ("^", "^") but now yields ("^^") as expected.
0.1
The tests are the documentation for the time-being.