Pyparsing

--------------------------
- NEW NOTATION SHORTCUT: ParserElement now accepts results names using
a notational shortcut, following the expression with the results name
in parentheses. So this:

stats = "AVE:" + realNum.setResultsName("average") + \
"MIN:" + realNum.setResultsName("min") + \
"MAX:" + realNum.setResultsName("max")

can now be written as this:

stats = "AVE:" + realNum("average") + \
"MIN:" + realNum("min") + \
"MAX:" + realNum("max")

The intent behind this change is to make it simpler to define results
names for significant fields within the expression, while keeping
the grammar syntax clean and uncluttered.

- Fixed bug when packrat parsing is enabled, with cached ParseResults
being updated by subsequent parsing. Reported on the pyparsing
wiki by Kambiz, thanks!

- Fixed bug in operatorPrecedence for unary operators with left
associativity, if multiple operators were given for the same term.

- Fixed bug in example simpleBool.py, corrected precedence of "and" vs.
"or" operations.

- Fixed bug in Dict class, in which keys were converted to strings
whether they needed to be or not. Have narrowed this logic to
convert keys to strings only if the keys are ints (which would
confuse __getitem__ behavior for list indexing vs. key lookup).

- Added ParserElement method setBreak(), which will invoke the pdb
module's set_trace() function when this expression is about to be
parsed.

- Fixed bug in StringEnd in which reading off the end of the input
string raises an exception - should match. Resolved while
answering a question for Shawn on the pyparsing wiki.

---------------------------
- Simplified constructor for ParseFatalException, to support common
exception construction idiom:
raise ParseFatalException, "unexpected text: 'Spanish Inquisition'"

- Added method getTokensEndLoc(), to be called from within a parse action,
for those parse actions that need both the starting *and* ending
location of the parsed tokens within the input text.

- Enhanced behavior of keepOriginalText so that named parse fields are
preserved, even though tokens are replaced with the original input
text matched by the current expression. Also, cleaned up the stack
traversal to be more robust. Suggested by Tim Arnold - thanks, Tim!

- Fixed subtle bug in which countedArray (and similar dynamic
expressions configured in parse actions) failed to match within Or,
Each, FollowedBy, or NotAny. Reported by Ralf Vosseler, thanks for
your patience, Ralf!

- Fixed Unicode bug in upcaseTokens and downcaseTokens parse actions,
scanString, and default debugging actions; reported (and patch submitted)
by Nikolai Zamkovoi, spasibo!

- Fixed bug when saving a tuple as a named result. The returned
token list gave the proper tuple value, but accessing the result by
name only gave the first element of the tuple. Reported by
Poromenos, nice catch!

- Fixed bug in makeHTMLTags/makeXMLTags, which failed to match tag
attributes with namespaces.

- Fixed bug in SkipTo when setting include=True, to have the skipped-to
tokens correctly included in the returned data. Reported by gunars on
the pyparsing wiki, thanks!

- Fixed typobug in OnceOnly.reset method, omitted self argument.
Submitted by eike welk, thanks for the lint-picking!

- Added performance enhancement to Forward class, suggested by
akkartik on the pyparsing Wiki discussion, nice work!

- Added optional asKeyword to Word constructor, to indicate that the
given word pattern should be matched only as a keyword, that is, it
should only match if it is within word boundaries.

- Added S-expression parser to examples directory.

- Added macro substitution example to examples directory.

- Added holaMundo.py example, excerpted from Marco Alfonso's blog -
muchas gracias, Marco!

- Modified internal cyclic references in ParseResults to use weakrefs;
this should help reduce the memory footprint of large parsing
programs, at some cost to performance (3-5%). Suggested by bca48150 on
the pyparsing wiki, thanks!

- Enhanced the documentation describing the vagaries and idiosyncrasies
of parsing strings with embedded tabs, and the impact on:
. parse actions
. scanString
. col and line helper functions
(Suggested by eike welk in response to some unexplained inconsistencies
between parsed location and offsets in the input string.)

- Cleaned up internal decorators to preserve function names,
docstrings, etc.

------------------------------
- Removed debugging print statement from QuotedString class. Sorry
for not stripping this out before the 1.4.4 release!

- A significant performance improvement, the first one in a while!
For my Verilog parser, this version of pyparsing is about double the
speed - YMMV.

- Added support for pickling of ParseResults objects. (Reported by
Jeff Poole, thanks Jeff!)

- Fixed minor bug in makeHTMLTags that did not recognize tag attributes
with embedded '-' or '_' characters. Also, added support for
passing expressions to makeHTMLTags and makeXMLTags, and used this
feature to define the globals anyOpenTag and anyCloseTag.

- Fixed error in alphas8bit, I had omitted the y-with-umlaut character.

- Added punc8bit string to complement alphas8bit - it contains all the
non-alphabetic, non-blank 8-bit characters.

- Added commonHTMLEntity expression, to match common HTML "ampersand"
codes, such as "<", ">", "&", " ", and """. This
expression also defines a results name 'entity', which can be used
to extract the entity field (that is, "lt", "gt", etc.). Also added
built-in parse action replaceHTMLEntity, which can be attached to
commonHTMLEntity to translate "<", ">", "&", " ", and
"&quot;" to "<", ">", "&", " ", and "'".

- Added example, htmlStripper.py, that strips HTML tags and scripts
from HTML pages. It also translates common HTML entities to their
respective characters.

-------------------------------
- Fixed traceParseAction decorator to also trap and record exception
returns from parse actions, and to handle parse actions with 0,
1, 2, or 3 arguments.

- Enhanced parse action normalization to support using classes as
parse actions; that is, the class constructor is called at parse
time and the __init__ function is called with 0, 1, 2, or 3
arguments. If passing a class as a parse action, the __init__
method must use one of the valid parse action parameter list
formats. (This technique is useful when using pyparsing to compile
parsed text into a series of application objects - see the new
example simpleBool.py.)

- Fixed bug in ParseResults when setting an item using an integer
index. (Reported by Christopher Lambacher, thanks!)

- Fixed whitespace-skipping bug, patch submitted by Paolo Losi -
grazie, Paolo!

- Fixed bug when a Combine contained an embedded Forward expression,
reported by cie on the pyparsing wiki - good catch!

- Fixed listAllMatches bug, when a listAllMatches result was
nested within another result. (Reported by don pasquale on
comp.lang.python, well done!)

- Fixed bug in ParseResults items() method, when returning an item
marked as listAllMatches=True

- Fixed bug in definition of cppStyleComment (and javaStyleComment)
in which '//' line comments were not continued to the next line
if the line ends with a '\'. (Reported by eagle-eyed Ralph
Corderoy!)

- Optimized re's for cppStyleComment and quotedString for better
re performance - also provided by Ralph Corderoy, thanks!

- Added new example, indentedGrammarExample.py, showing how to
define a grammar using indentation to show grouping (as Python
does for defining statement nesting). Instigated by an e-mail
discussion with Andrew Dalke, thanks Andrew!

- Added new helper operatorPrecedence (based on e-mail list discussion
with Ralph Corderoy and Paolo Losi), to facilitate definition of
grammars for expressions with unary and binary operators. For
instance, this grammar defines a 6-function arithmetic expression
grammar, with unary plus and minus, proper operator precedence,and
right- and left-associativity:

expr = operatorPrecedence( operand,
[("!", 1, opAssoc.LEFT),
("^", 2, opAssoc.RIGHT),
(oneOf("+ -"), 1, opAssoc.RIGHT),
(oneOf("* /"), 2, opAssoc.LEFT),
(oneOf("+ -"), 2, opAssoc.LEFT),]
)

Also added example simpleArith.py and simpleBool.py to provide
more detailed code samples using this new helper method.

- Added new helpers matchPreviousLiteral and matchPreviousExpr, for
creating adaptive parsing expressions that match the same content
as was parsed in a previous parse expression. For instance:

first = Word(nums)
matchExpr = first + ":" + matchPreviousLiteral(first)

will match "1:1", but not "1:2". Since this matches at the literal
level, this will also match the leading "1:1" in "1:10".

In contrast:

first = Word(nums)
matchExpr = first + ":" + matchPreviousExpr(first)

will *not* match the leading "1:1" in "1:10"; the expressions are
evaluated first, and then compared, so "1" is compared with "10".

- Added keepOriginalText parse action. Sometimes pyparsing's
whitespace-skipping leaves out too much whitespace. Adding this
parse action will restore any internal whitespace for a parse
expression. This is especially useful when defining expressions
for scanString or transformString applications.

- Added __add__ method for ParseResults class, to better support
using Python sum built-in for summing ParseResults objects returned
from scanString.

- Added reset method for the new OnlyOnce class wrapper for parse
actions (to allow a grammar to be used multiple times).

- Added optional maxMatches argument to scanString and searchString,
to short-circuit scanning after 'n' expression matches are found.

------------------------------
- Fixed implementation of multiple parse actions for an expression
(added in 1.4.2).
. setParseAction() reverts to its previous behavior, setting
one (or more) actions for an expression, overwriting any
action or actions previously defined
. new method addParseAction() appends one or more parse actions
to the list of parse actions attached to an expression
Now it is harder to accidentally append parse actions to an
expression, when what you wanted to do was overwrite whatever had
been defined before. (Thanks, Jean-Paul Calderone!)

- Simplified interface to parse actions that do not require all 3
parse action arguments. Very rarely do parse actions require more
than just the parsed tokens, yet parse actions still require all
3 arguments including the string being parsed and the location
within the string where the parse expression was matched. With this
release, parse actions may now be defined to be called as:
. fn(string,locn,tokens) (the current form)
. fn(locn,tokens)
. fn(tokens)
. fn()
The setParseAction and addParseAction methods will internally decorate
the provided parse actions with compatible wrappers to conform to
the full (string,locn,tokens) argument sequence.

- REMOVED SUPPORT FOR RETURNING PARSE LOCATION FROM A PARSE ACTION.
I announced this in March, 2004, and gave a final warning in the last
release. Now you can return a tuple from a parse action, and it will
be treated like any other return value (i.e., the tuple will be
substituted for the incoming tokens passed to the parse action,
which is useful when trying to parse strings into tuples).

- Added setFailAction method, taking a callable function fn that
takes the arguments fn(s,loc,expr,err) where:
. s - string being parsed
. loc - location where expression match was attempted and failed
. expr - the parse expression that failed
. err - the exception thrown
The function returns no values. It may throw ParseFatalException
if it is desired to stop parsing immediately.
(Suggested by peter21081944 on wikispaces.com)

- Added class OnlyOnce as helper wrapper for parse actions. OnlyOnce
only permits a parse action to be called one time, after which
all subsequent calls throw a ParseException.

- Added traceParseAction decorator to help debug parse actions.
Simply insert "traceParseAction" ahead of the definition of your
parse action, and each invocation will be displayed, along with
incoming arguments, and returned value.

- Fixed bug when copying ParserElements using copy() or
setResultsName(). (Reported by Dan Thill, great catch!)

- Fixed bug in asXML() where token text contains <, >, and &
characters - generated XML now escapes these as &lt;, &gt; and
&amp;. (Reported by Jacek Sieka, thanks!)

- Fixed bug in SkipTo() when searching for a StringEnd(). (Reported
by Pete McEvoy, thanks Pete!)

- Fixed "except Exception" statements, the most critical added as part
of the packrat parsing enhancement. (Thanks, Erick Tryzelaar!)

- Fixed end-of-string infinite looping on LineEnd and StringEnd
expressions. (Thanks again to Erick Tryzelaar.)

- Modified setWhitespaceChars to return self, to be consistent with
other ParserElement modifiers. (Suggested by Erick Tryzelaar.)

- Fixed bug/typo in new ParseResults.dump() method.

- Fixed bug in searchString() method, in which only the first token of
an expression was returned. searchString() now returns a
ParseResults collection of all search matches.

- Added example program removeLineBreaks.py, a string transformer that
converts text files with hard line-breaks into one with line breaks
only between paragraphs.

- Added example program listAllMatches.py, to illustrate using the
listAllMatches option when specifying results names (also shows new
support for passing lists to oneOf).

- Added example program linenoExample.py, to illustrate using the
helper methods lineno, line, and col, and returning objects from a
parse action.

- Added example program parseListString.py, to which can parse the
string representation of a Python list back into a true list. Taken
mostly from my PyCon presentation examples, but now with support
for tuple elements, too!

-------------------------------------------
- Significant speedup from memoizing nested expressions (a technique
known as "packrat parsing"), thanks to Chris Lesniewski-Laas! Your
mileage may vary, but my Verilog parser almost doubled in speed to
over 600 lines/sec!

This speedup may break existing programs that use parse actions that
have side-effects. For this reason, packrat parsing is disabled when
you first import pyparsing. To activate the packrat feature, your
program must call the class method ParserElement.enablePackrat(). If
your program uses psyco to "compile as you go", you must call
enablePackrat before calling psyco.full(). If you do not do this,
Python will crash. For best results, call enablePackrat() immediately
after importing pyparsing.

- Added new helper method countedArray(expr), for defining patterns that
start with a leading integer to indicate the number of array elements,
followed by that many elements, matching the given expr parse
expression. For instance, this two-liner:
wordArray = countedArray(Word(alphas))
print wordArray.parseString("3 Practicality beats purity")[0]
returns the parsed array of words:
['Practicality', 'beats', 'purity']
The leading token '3' is suppressed, although it is easily obtained
from the length of the returned array.
(Inspired by e-mail discussion with Ralf Vosseler.)

- Added support for attaching multiple parse actions to a single
ParserElement. (Suggested by Dan "Dang" Griffith - nice idea, Dan!)

- Added support for asymmetric quoting characters in the recently-added
QuotedString class. Now you can define your own quoted string syntax
like "<<This is a string in double angle brackets.>>". To define
this custom form of QuotedString, your code would define:
dblAngleQuotedString = QuotedString('<<',endQuoteChar='>>')
QuotedString also supports escaped quotes, escape character other
than '\', and multiline.

- Changed the default value returned internally by Optional, so that
None can be used as a default value. (Suggested by Steven Bethard -
I finally saw the light!)

- Added dump() method to ParseResults, to make it easier to list out
and diagnose values returned from calling parseString.

- A new example, a search query string parser, submitted by Steven
Mooij and Rudolph Froger - a very interesting application, thanks!

- Added an example that parses the BNF in Python's Grammar file, in
support of generating Python grammar documentation. (Suggested by
J H Stovall.)

- A new example, submitted by Tim Cera, of a flexible parser module,
using a simple config variable to adjust parsing for input formats
that have slight variations - thanks, Tim!

- Added an example for parsing Roman numerals, showing the capability
of parse actions to "compile" Roman numerals into their integer
values during parsing.

- Added a new docs directory, for additional documentation or help.
Currently, this includes the text and examples from my recent
presentation at PyCon.

- Fixed another typo in CaselessKeyword, thanks Stefan Behnel.

- Expanded oneOf to also accept tuples, not just lists. This really
should be sufficient...

- Added deprecation warnings when tuple is returned from a parse action.
Looking back, I see that I originally deprecated this feature in March,
2004, so I'm guessing people really shouldn't have been using this
feature - I'll drop it altogether in the next release, which will
allow users to return a tuple from a parse action (which is really
handy when trying to reconstuct tuples from a tuple string
representation!).