Nmfu

This is a major feature release, vastly improving NMFU's capability for parsing binary protocols, in addition to fixing a number of bugs. There's also now a proper documentation site for NMFU, available at [nmfu.rtfd.io](https://nmfu.rtfd.io).

New Features

- Expanded type system: (ea5e297)
- Integers can now have a custom size and signedness: `out int{size 4, unsigned} wow;`
- Strings can now be unterminated: `out unterminated str[56] look_no_nulls;`
- Conditional blocks: `if`/`elif`/`else` statements now exist (be41210, f3f71de).
- Expanded expressions:
- Boolean expressions: `||` and `&&`, as well as comparison operators (baa2ba1, 3369969)
- Better math: unary not and negation (5f24078), string length and indexing (353f391) and bitwise logic/bitshifts (6936042).
- Better integer literals; now supporting hexadecimal and binary: `0b0110`, `-0x54` (1e61988)
- Binary matches (8b2f09b, e202041):
- Strings and regexes can now be made "binary", where instead of them having characters they instead only contain hex.
- For example, `"f0 0ffa"b` or `b/ff+ (3456|56 77{2-3})/`.
- This also adds support for `\xHH` escapes within normal strings.
- Character constants now support escape sequences (45196d6, 1cdd324)
- Improved debugging tools for working on NMFU itself:
- Option for dumping the parse tree for debugging (1b7242a).
- DFA-dumper can now be asked to show all literals as hex codepoints (3450fa5)
- Gave conditional actions proper debug names (151eaae)
- Added dumper for viewing the `DTAG` tree (c45b4d2)
- New optimizer that tries to shortcircuit fallthrough transitions if possible (3043adc, db126e2)

Bugs Fixed

- Better ambiguity detection and resolution for chaining matches, especially those with inverted matches (1026f83).
- Properly handle actions after certain blocks when there is nothing following them (c5f1806).
- Correct short-circuit behaviour for action side-effects (such as `outofspace` handling) (bbda784).
- More accurate line/column information for errors concerning regexes (92cd8f2).
- Ensure chaining DFAs doesn't drop error handling-transitions in certain circumstances (64e89f2)
- Fix `DFA.dfs()` not properly considering `MAY_GOTO_TARGET`-type actions, causing the optimizer to mistakenly drop states (ca72b3b)
- Fixed loop body start actions being ignored (ccb54a3)
- Macros now bind arguments in a sane way, fixing various problems with nesting them. (8f87354, 6191b42)
- Fallthrough loop detection now considers multi-state cycles. (222d971)
- Action-only conditional nodes now properly propagate subsequent actions (ad3a917)
- Foreach nodes only apply their actions to non-fallthrough transitions to ensure they only run once per character (709d314)
- Rewrote the majority of the loop implementation to make it more compatible with action side-effects (5b520bb)
- `DTAG` system now does more checks for garbage-collection and uses weakrefs to be slightly more memory-efficient (e5d58f3)

Infrastructure Changes

- Proper documentation, powered by mkdocs (ce5429c) as well as a basic tutorial (a1ea939, 3af8af2, 3897641)
- New example showcasing binary features which parses `TripUpdate` messages from a GTFS Realtime feed (f4a2edb, e7c8b1d)
- Demonstration of more advanced string manipulation in the RDF example (92ff838, 5fa891e)
- Full integration tests now will run the generated code through an actual compiler to check for correctness. While they're skipped if one isn't found, please make sure any potential patches are tested with one installed. (acbff31)
- Certain integration tests now check for both compilation success and correct matching (a2fb61a)
- Code coverage is now reported in CI (2b9152f)
- More tests, both for new and old features. (lots)

This is another early preview of some more new features in the upcoming 0.4.0 release, as well as some fixes for regressions in 0.4.0a1.

New Features

- More math operators: unary not and negation (5f24078), string length and indexing (353f391) and bitwise logic/bitshifts (6936042).
- Character constants now support escape sequences (45196d6)
- Expanded type system: (ea5e297)
- Integers can now have a custom size and signedness: `out int{size 4, unsigned} wow;`
- Strings can now be unterminated: `out unterminated str[56] look_no_nulls;`
- Better integer literals; now supporting hexadecimal and binary: `0b0110`, `-0x54` (1e61988)
- Improved NMFU-debugging:
- DFA-dumper can now be asked to show all literals as hex codepoints (3450fa5)
- Gave conditional actions proper debug names (151eaae)
- Added dumper for viewing the `DTAG` tree (c45b4d2)
- New optimizer that tries to shortcircuit fallthrough transitions if possible (3043adc, db126e2)

Bugs Fixed

- Ensure chaining DFAs doesn't drop error handling-transitions in certain circumstances (64e89f2)
- Fix `DFA.dfs()` not properly considering `MAY_GOTO_TARGET`-type actions, causing the optimizer to mistakenly drop states (ca72b3b)
- Fixed loop body start actions being ignored (ccb54a3)
- Macros now bind arguments in a sane way, fixing various problems with nesting them. (8f87354, 6191b42)
- Fallthrough loop detection now considers multi-state cycles. (222d971)
- Action-only conditional nodes now properly propagate subsequent actions (ad3a917)
- Foreach nodes only apply their actions to non-fallthrough transitions to ensure they only run once per character (709d314)
- Rewrote the majority of the loop implementation to make it more compatible with action side-effects (5b520bb)
- `DTAG` system now does more checks for garbage-collection and uses weakrefs to be slightly more memory-efficient (e5d58f3)

Infrastructure Changes

- New test showcasing binary features which parses `TripUpdate` messages from a GTFS Realtime feed (f4a2edb, e7c8b1d)
- Demonstration of more advanced string manipulation in the RDF example (92ff838, 5fa891e)
- Full integration tests now will run the generated code through an actual compiler to check for correctness. While they're skipped properly if one isn't found, please make sure any potential patches are tested with one installed. (acbff31)

This is an early preview of some of the new features in the upcoming 0.4.0 release.

New Features

- Conditional blocks: `if`/`elif`/`else` statements now exist (be41210, f3f71de).
- Boolean expressions: `||` and `&&`, as well as comparison operators (baa2ba1, 3369969)
- Binary matches (8b2f09b, e202041):
- Strings and regexes can now be made "binary", where instead of them having characters they instead only contain hex.
- For example, `"f0 0ffa"b` or `b/ff+ (3456|56 77{2-3})/`.
- This also adds support for `\xHH` escapes within normal strings.
- Option for dumping the parse tree for debugging (1b7242a).

Bugs Fixed

- Better ambiguity detection and resolution for chaining matches, especially those with inverted matches (1026f83).
- Properly handle actions after certain blocks when there is nothing following them (c5f1806).
- Correct short-circuit behaviour for action side-effects (such as `outofspace` handling) (bbda784).
- More accurate line/column information for errors concerning regexes (92cd8f2).

Infrastructure Changes

- Certain integration tests now check for both compilation success and correct matching (a2fb61a)
- Code coverage is now reported in CI (2b9152f)
- More tests, both for new and old features. (lots)

This is a minor release that tweaks a few things in code generation.

New Features

- `jpto_` labels are only emitted when used in outputted code, preventing `-Wunused-label` warnings. (1b93e4b)
- Allow customizing the output format of debugging graphs. (5005bae)

This is a feature release which adds a bunch of miscellaneous features, in addition to changing the public-facing API.

New Features

- New API: (cbd45c4)
- Instead of taking in single characters, the `_feed` function now takes in blocks of characters at once. The function can also
optionally report how many characters it read in case of early finishes, to aid in chaining parsers.
- Instead of having a parameter for `is_end`, there is now an optional separate `_end` function for handling EOF detection, enabled
by the `-feof-support` flag.
- These changes are designed to improve generated parser efficiency.
- Fallthrough semantics: (cce57c4, af6728f)
- Instead of `else` matches in case nodes and catch blocks consuming characters, they now forward the error/else to the first match
in their block.
- This makes error handling more robust, as well as removing a lot of the uses for `some_var += [$last]`, simplifying parsers.
- Revamped regexes: (50f219c, 5603d44)
- Operator precedence is now more consistent with typical regexes, so `(option1|option2)` now works correctly.
- Repetition syntax (`\d{4-6}`) is now supported.
- Foreach statement: (9cda4ac)
- Parsers can now execute statements on each character matched by some region.
- This greatly simplifies things like reading numbers:

foreach {
\d+\;
} do {
num = [num * 10 + ($last - '0')];
}

- Macros can now take arguments. (07e4a04)
- Character constants can now be used in math expressions. (fc93c91)

Bugs Fixed

- Fixed problem where having a wait match after a try except could break due to incomplete error-handling redirection. (87de753)
- Fixed internal error with `allow_replace` in `DFState.transition` which could hide ambiguous transitions. (0e1ad3b)
- Fixed problems with incomplete `else` handling on case nodes with regex-matches with inverted character sets/wildcards. (417fb41, 3fa4d93)

Infrastructure Changes

- NMFU now also builds AppImages (57ff2e7, 193910a)
- The test suite has been thoroughly expanded, and now covers a lot more. (lots)
- There is now an automated version update script. (2556d66)

This is another early preview of some more changes in the upcoming 0.3.0 release.

New Features

- Added a mode for making the `start` pointer indirect, allowing `_feed` to report how much of its input it consumed. (e589c6f)
- Changed regex grammar to more closely match other regex syntax (by fixing the precedence of the `|` operator.) (50f219c)
- Added support for regex repetition matches (5603d44)
- Added a new block syntax, `foreach`, which can execute statements on each character consumed by a block. (9cda4ac)
- Added character constants in math expressions. (fc93c91)

Bugs Fixed

- Various problems with the new fallthrough transitions from 0.3.0a1. (4203c2e, 68e95a3)
- Fixed problem where having a wait match after a try except could break due to incomplete error-handling redirection. (87de753)
- Fixed internal error with `allow_replace` in `DFState.transition` which could hide ambiguous transitions. (0e1ad3b)
- Fixed problems with incomplete `else` handling on case nodes with regex-matches with inverted character sets/wildcards. (417fb41, 3fa4d93)
- Fix end matches, which were broken in 0.3.0a1. (4e3b6db, 4f4dba9)

Infrastructure Changes

- The test suite has been thoroughly expanded, and now covers a lot more.
- There is now an automated version update script.