Guidance

What's Changed

Happy new year! This release contains a significant overhaul of many elements of guidance, including:

- **Visualization Overhaul**: Guidance's in-line visualizations have been rewritten from scratch. TL;DR: We added a lot of code to support better model metrics and visualization. Check it out:


- **Significantly Faster, Rust-based Grammar Engine**: Guidance's core grammar processing has been fully migrated to the [llguidance](https://github.com/guidance-ai/llguidance) Rust library. Our grammar processing is now significantly faster -- state of the art across frameworks -- and fixes some key, subtle bugs in the earlier processing engine. All constrained decoding code will now run much faster.

- **Increased JSON schema coverage**: We've spent a lot of time focusing on JSON schema support in guidance, and now have leading coverage of JSON schema constraint types via guidance's ``json`` function.

- And lots more general quality of life improvements! See below for full list of PRs and changelogs. Thank you to all the existing and new contributors that made this possible!

---------------

* [WIP] Replace EarleyParser with lexeme-based rust implementation by hudson-ai in https://github.com/guidance-ai/guidance/pull/951
* Update README.md with new grammar-enabled endpoint details (Azure AI) by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/998
* [Test] Enable Azure Guidance endpoints in test matrix by riedgar-ms in https://github.com/guidance-ai/guidance/pull/999
* [Test] Move Azure Guidance in CI by riedgar-ms in https://github.com/guidance-ai/guidance/pull/1000
* [Bug] del `f._self_call_placeholder_` even when calling `f` raises exception by hudson-ai in https://github.com/guidance-ai/guidance/pull/994
* [mypy] Improve types in _grammar.py by riedgar-ms in https://github.com/guidance-ai/guidance/pull/1002
* [Build] CI & Notebook Workaround by riedgar-ms in https://github.com/guidance-ai/guidance/pull/1005
* [Feature] Allow `oneOf` in JSON schemas (with limited support) by hudson-ai in https://github.com/guidance-ai/guidance/pull/982
* Use `json_string` kwarg from `llguidance 0.1.7`to properly escape json string patterns by hudson-ai in https://github.com/guidance-ai/guidance/pull/1008
* Remove `Placeholder` and `replace_grammar_node` in favor of pointer-like container object by hudson-ai in https://github.com/guidance-ai/guidance/pull/1007
* [Feature] Support `required` properties in JSON schemas by hudson-ai in https://github.com/guidance-ai/guidance/pull/1009
* [Housekeeping] Clean up Enums to make JSON features more transparent by hudson-ai in https://github.com/guidance-ai/guidance/pull/1011
* [Doc] Fix typos in README and tutorials notebooks by gfesatidis in https://github.com/guidance-ai/guidance/pull/1012
* fix test_ll for new llguidance with working advance_parser() by mmoskal in https://github.com/guidance-ai/guidance/pull/1013
* [Bug] Ensure `grammar.match` raises appropriately on incomplete input by hudson-ai in https://github.com/guidance-ai/guidance/pull/1016
* Support boolean JSON schemas by hudson-ai in https://github.com/guidance-ai/guidance/pull/1015
* `overload` the `guidance` decorator in order to improve static type-checking by hudson-ai in https://github.com/guidance-ai/guidance/pull/1014
* JSON string 'format' kwarg by hudson-ai in https://github.com/guidance-ai/guidance/pull/984
* Strip anchors from JSON regexes by hudson-ai in https://github.com/guidance-ai/guidance/pull/1027
* JSON reserved keywords by hudson-ai in https://github.com/guidance-ai/guidance/pull/1032
* "required" JSON keys are validated against "additionalProperties" if they are missing from "properties" by hudson-ai in https://github.com/guidance-ai/guidance/pull/1029
* [Bug] Make `_self_call_placeholder_` thread-safe by hudson-ai in https://github.com/guidance-ai/guidance/pull/1034
* add test for "nullable bug" by mmoskal in https://github.com/guidance-ai/guidance/pull/1025
* Make the tests pass with larger tokenizers by mmoskal in https://github.com/guidance-ai/guidance/pull/1037
* [Feature] Support for JSON `minimum` , `maximum`, `exclusiveMinimum`, and `exclusiveMaximum` by hudson-ai in https://github.com/guidance-ai/guidance/pull/1023
* [Feature] Allow `guidance` to decorate methods by hudson-ai in https://github.com/guidance-ai/guidance/pull/1035
* [Bug] [Feature] Fix issue 1039 by exposing `And`, `Not` regex operators from `llguidance` by hudson-ai in https://github.com/guidance-ai/guidance/pull/1043
* [WIP] Try to get PR Gate working again... by hudson-ai in https://github.com/guidance-ai/guidance/pull/1048
* [Feature] Enable user-passed `separators` keyword to `guidance.json` by hudson-ai in https://github.com/guidance-ai/guidance/pull/1044
* fix typos in README by Saibo-creator in https://github.com/guidance-ai/guidance/pull/1056
* [Feature] Expand JSON `$ref` resolution by hudson-ai in https://github.com/guidance-ai/guidance/pull/1061
* [JSON] Raise exceptions when "sibling" keywords are unhandled by hudson-ai in https://github.com/guidance-ai/guidance/pull/1063
* [Feature] Allow json-loads-able strings to be passed as schema by hudson-ai in https://github.com/guidance-ai/guidance/pull/1028
* [Hotfix] Echo forced to false on terminal detection. by nopdive in https://github.com/guidance-ai/guidance/pull/1067
* [Keeping House] Split JSON tests into multiple files by hudson-ai in https://github.com/guidance-ai/guidance/pull/1072
* [Feature] [JSON] Broader support of `allOf` JSON keyword by hudson-ai in https://github.com/guidance-ai/guidance/pull/1070
* [BUG] Reset role blocks when `model.reset` is called by hudson-ai in https://github.com/guidance-ai/guidance/pull/1073
* [Optimization] Advance parser concurrently with model forward pass by hudson-ai in https://github.com/guidance-ai/guidance/pull/1065
* Update README.md by culmat in https://github.com/guidance-ai/guidance/pull/1045
* [Feature] Add qwen2.5 chat templates by xruifan in https://github.com/guidance-ai/guidance/pull/1066
* Expose fastforward and backtrack llguidance flags by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/1075
* [Bug] Fix for environment detection in barebones environments. by nopdive in https://github.com/guidance-ai/guidance/pull/1076
* Update setup.py to limit llguidance version by hudson-ai in https://github.com/guidance-ai/guidance/pull/1080
* update to llguidance 0.4.1 by mmoskal in https://github.com/guidance-ai/guidance/pull/1081
* fix current failures in pytest when run locally by paulbkoch in https://github.com/guidance-ai/guidance/pull/1083
* Force Transformers engine to return float logits by JC1DA in https://github.com/guidance-ai/guidance/pull/1086
* Bump runners from macos-12 to macos-14 by hudson-ai in https://github.com/guidance-ai/guidance/pull/1088
* update llguidance to 0.5.0 by mmoskal in https://github.com/guidance-ai/guidance/pull/1087
* Use `llguidance` JSON grammar by hudson-ai in https://github.com/guidance-ai/guidance/pull/1085
* Upgrade llguidance to `0.5.1rc0` by hudson-ai in https://github.com/guidance-ai/guidance/pull/1090
* Interactivity Overhaul (User Interface & Model Instrumentation & Network Comms) by nopdive in https://github.com/guidance-ai/guidance/pull/1054

New Contributors
* gfesatidis made their first contribution in https://github.com/guidance-ai/guidance/pull/1012
* Saibo-creator made their first contribution in https://github.com/guidance-ai/guidance/pull/1056
* culmat made their first contribution in https://github.com/guidance-ai/guidance/pull/1045
* xruifan made their first contribution in https://github.com/guidance-ai/guidance/pull/1066
* JC1DA made their first contribution in https://github.com/guidance-ai/guidance/pull/1086

**Full Changelog**: https://github.com/guidance-ai/guidance/compare/0.1.16...0.2.0

What's Changed
* Replace Python-based EarleyParser with lexeme-based rust implementation by hudson-ai & mmoskal in https://github.com/guidance-ai/guidance/pull/951

This pre-release version of guidance uses a new Rust-based Earley lexer + parser that offers significant speedups and bugfixes to guidance grammar parsing. See PR https://github.com/guidance-ai/guidance/pull/951 for full details of the optimizations. Expect some changes from current Guidance (v0.1.16) behavior during pre-release.

**Full Changelog**: https://github.com/guidance-ai/guidance/compare/0.1.16...0.2.0rc1

What's Changed
* Allow generating arbitrary (schemaless) JSON by hudson-ai in https://github.com/guidance-ai/guidance/pull/892
* [Bug] Fix Llama3ChatTemplate by jjkjkj in https://github.com/guidance-ai/guidance/pull/884
* Use python's `re` module instead of `pyformlang` to build regex ASTs by hudson-ai in https://github.com/guidance-ai/guidance/pull/854
* [Feature] Add Support for Phi-3-small by ambisinister in https://github.com/guidance-ai/guidance/pull/935
* [Feature] Constant elements in JSON schemas by wjn0 in https://github.com/guidance-ai/guidance/pull/886
* [Bug] Fix for LlamaCpp tokeniser prepending spaces by riedgar-ms in https://github.com/guidance-ai/guidance/pull/903
* New Benchmarking system by nopdive in https://github.com/guidance-ai/guidance/pull/843
* Mistral chat template integrated by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/919
* [Bug] Passing through kwargs to `TransformersTokenizer` by riedgar-ms in https://github.com/guidance-ai/guidance/pull/922
* [Feature] Add extra support to JSON string schema by riedgar-ms in https://github.com/guidance-ai/guidance/pull/927
* Update protobuf definitions for remote list_append fix by nking-1 in https://github.com/guidance-ai/guidance/pull/937
* Downgrade protobuf required version using protoc 25.3 by nking-1 in https://github.com/guidance-ai/guidance/pull/946
* [Bug] Fix regex for urls in image function by kklemon in https://github.com/guidance-ai/guidance/pull/928
* Dedent fixes for Python 3.12 by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/959
* Refector `TransformersTokenizer` and change fallback behavior for `byte_tokens` by hudson-ai in https://github.com/guidance-ai/guidance/pull/973
* Fix Transformer models with bos_token null by Mihaiii in https://github.com/guidance-ai/guidance/pull/990

New Contributors
* nopdive made their first contribution in https://github.com/guidance-ai/guidance/pull/843
* jjkjkj made their first contribution in https://github.com/guidance-ai/guidance/pull/884
* wjn0 made their first contribution in https://github.com/guidance-ai/guidance/pull/886
* kklemon made their first contribution in https://github.com/guidance-ai/guidance/pull/928
* ambisinister made their first contribution in https://github.com/guidance-ai/guidance/pull/935
* Mihaiii made their first contribution in https://github.com/guidance-ai/guidance/pull/990

**Full Changelog**: https://github.com/guidance-ai/guidance/compare/0.1.15...0.1.16

What's Changed
* Support for Anthropic Claude 3 and Phi-3 models by Harsha-Nori
* [Feature] Metrics for Grammarless models by riedgar-ms in https://github.com/guidance-ai/guidance/pull/817
* Remove Instruct/Chat versions of models & introduce a new ChatTemplate API by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/820
* Fix set_attribute by slundberg in https://github.com/guidance-ai/guidance/pull/835
* Fix for unstable tokenization schemes by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/830
* Support null captures (Fix 826) by slundberg in https://github.com/guidance-ai/guidance/pull/834
* Allow capture of raw functions by slundberg in https://github.com/guidance-ai/guidance/pull/837
* [Build] Significant notebook testing updates by riedgar-ms in https://github.com/guidance-ai/guidance/pull/825
* [Docs] ReadTheDocs Now Works! by riedgar-ms in https://github.com/guidance-ai/guidance/pull/829
* add initial support for azure guidance server API by paulbkoch in https://github.com/guidance-ai/guidance/pull/827
* [Build] Adjust llama-cpp-python versions by riedgar-ms in https://github.com/guidance-ai/guidance/pull/832
* [Test] Fix for LlamaCpp tests by paulbkoch in https://github.com/guidance-ai/guidance/pull/839
* Remove default max token limit by slundberg in https://github.com/guidance-ai/guidance/pull/836
* [Build] General Maintenance by riedgar-ms in https://github.com/guidance-ai/guidance/pull/842
* Updated readme by nking-1 in https://github.com/guidance-ai/guidance/pull/810


**Full Changelog**: https://github.com/guidance-ai/guidance/compare/0.1.14...0.1.15

This release brings a series of new features, bug fixes, and changes, including:

- Improved compatibility for loading many `transformers` models, including newly tested support for `Phi-3`
- New capabilities in the `guidance.json` function for generating json, including early support for pydantic schemas.
- Support for AzureAI models in new `guidance.models.AzureAIStudio*` classes
- Significant dependency reductions on the default guidance installation
- Small grammar optimizations
- Many improvements to the infrastructure, testing, and release pipelines :)

What's Changed
* replace catches of the ImportError exception with the more specific ModuleNotFoundError exception by paulbkoch in https://github.com/guidance-ai/guidance/pull/733
* fix for issue 660 where the call to llama_batch_free is failing during process shutdown by paulbkoch in https://github.com/guidance-ai/guidance/pull/734
* Fix 713 for mistral models in transformers by slundberg in https://github.com/guidance-ai/guidance/pull/740
* Update JSON API and example notebook by riedgar-ms in https://github.com/guidance-ai/guidance/pull/699
* remove redundant options from select() by mmoskal in https://github.com/guidance-ai/guidance/pull/712
* Add support for `prefixItems`, `minItems`, and `maxItems` for `JSON` array generation by hudson-ai in https://github.com/guidance-ai/guidance/pull/732
* [Docs] Expand documentation by riedgar-ms in https://github.com/guidance-ai/guidance/pull/750
* Generation with `pydantic` schemas by hudson-ai in https://github.com/guidance-ai/guidance/pull/724
* First stab at minimal mypy implementation by ryanpeach in https://github.com/guidance-ai/guidance/pull/738
* remove openai hard dependency by slundberg in https://github.com/guidance-ai/guidance/pull/763
* from guidance import guidance by ryanpeach in https://github.com/guidance-ai/guidance/pull/759
* Run black on codebase by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/765
* add rust bindings by paulbkoch in https://github.com/guidance-ai/guidance/pull/736
* fix Transformers tokenizer issues when models have extended vocabulary and/or special tokens by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/766
* [Install] Rework optional imports by riedgar-ms in https://github.com/guidance-ai/guidance/pull/767
* Update _llama_cpp.py based on new contract of llama_get_logits by czoccolo in https://github.com/guidance-ai/guidance/pull/769
* [Bug] Missing support for AzureAI by riedgar-ms in https://github.com/guidance-ai/guidance/pull/770
* [Doc] Add developer notes by riedgar-ms in https://github.com/guidance-ai/guidance/pull/774
* [Feature] Improved JSON Schema support by riedgar-ms in https://github.com/guidance-ai/guidance/pull/787
* [Bug] Fix Regex for identifying OpenAI models by alexander-brady in https://github.com/guidance-ai/guidance/pull/778
* Fix models like phi-3 which have a mismatched tokenizer definition and model output tensor size by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/792
* [Feature] Add `temperature` to JSON generator by riedgar-ms in https://github.com/guidance-ai/guidance/pull/784
* [Feature] Support for Azure AI Studio by riedgar-ms in https://github.com/guidance-ai/guidance/pull/779
* Add guard for running build from setup.py, redirecting users to pip by Harsha-Nori in https://github.com/guidance-ai/guidance/pull/805
* *Many* testing and CI improvements by riedgar-ms

New Contributors
* mmoskal made their first contribution in https://github.com/guidance-ai/guidance/pull/712
* ryanpeach made their first contribution in https://github.com/guidance-ai/guidance/pull/738
* czoccolo made their first contribution in https://github.com/guidance-ai/guidance/pull/769
* alexander-brady made their first contribution in https://github.com/guidance-ai/guidance/pull/778

**Full Changelog**: https://github.com/guidance-ai/guidance/compare/0.1.13...0.1.14

What's Changed
* Fixed PyPi publishing
* Workflow for GPU Runner by riedgar-ms in https://github.com/guidance-ai/guidance/pull/694
* Add Llama to model test matrix by riedgar-ms in https://github.com/guidance-ai/guidance/pull/703


**Full Changelog**: https://github.com/guidance-ai/guidance/compare/0.1.12...0.1.13