Keras-nlp

Summary
* Mistral fixes for dtype and memory usage. https://github.com/keras-team/keras-nlp/issues/1458

What's Changed
* Fix Mistral memory consumption with JAX and default dtype bug by tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1460
* Version bump for dev release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1474


**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.8.1...v0.8.2.dev0

Minor fixes to Kaggle Gemma assets.

What's Changed
* Update to the newest version of Gemma on Kaggle by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1454
* Dev release 0.8.1.dev0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1456
* 0.8.1 version bump by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1457


**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.8.0...v0.8.1

The 0.8.0 release focuses on generative LLM features in KerasNLP.

Summary
* Added the `Mistral` and `Gemma` models.
* Allow passing `dtype` directly to backbone and task constructors.
* Add a settable `sequence_length` property to all preprocessing layers.
* Added `enable_lora()` to the backbone class for parameter efficient fine-tuning.
* Added layer attributes to backbone models for easier access to model internals.
* Added `AlibiBias` layer.

python
Pass dtype to a model.
causal_lm = keras_nlp.MistralCausalLM.from_preset(
"mistral_instruct_7b_en",
dtype="bfloat16"
)
Settable sequence length property.
causal_lm.preprocessor.sequence_length = 128
Lora API.
causal_lm.enable_lora(rank=4)
Easy layer attributes.
for layer in causal_lm.backbone.transformer_layers:
print(layer.count_params())


What's Changed
* Fix test for recent keras 3 change by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1400
* Pass less state to jax generate function by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1398
* Add llama tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1401
* Add Bloom Model by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1382
* Try fixing tests by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1411
* Revert "Pass less state to jax generate function (1398)" by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1412
* Bloom tokenizer by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1403
* Update black formatting by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1415
* Add Alibi bias layer by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1404
* Pin to `tensorflow-hub 0.16.0` to fix CI error by sampathweb in https://github.com/keras-team/keras-nlp/pull/1420
* Update TF Text and remove TF Hub deps by sampathweb in https://github.com/keras-team/keras-nlp/pull/1423
* Pin Jax Version in GPU CI by sampathweb in https://github.com/keras-team/keras-nlp/pull/1430
* Add Bloom preprocessor by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1424
* Add layer attributes for all functional models by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1421
* Allow setting dtype per model by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1431
* Add a Causal LM model for Mistral by tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1429
* Fix bart by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1434
* Add a settable property for sequence_length by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1437
* Add dependabot to update GH Actions and Python dependencies by pnacht in https://github.com/keras-team/keras-nlp/pull/1380
* Bump the github-actions group with 1 update by dependabot in https://github.com/keras-team/keras-nlp/pull/1438
* Add 7B presets for Mistral by tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1436
* Update byte_pair_tokenizer.py to close merges file properly by divyashreepathihalli in https://github.com/keras-team/keras-nlp/pull/1440
* bump version to 0.8 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1441
* Update our sampler documentation to reflect usage by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1444
* Add Gemma model by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1448
* Version bump for dev release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1449
* Version bump to 0.8.0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1450

New Contributors
* dependabot made their first contribution in https://github.com/keras-team/keras-nlp/pull/1438
* divyashreepathihalli made their first contribution in https://github.com/keras-team/keras-nlp/pull/1440

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.7.0...v0.8.0

This release integrates KerasNLP and [Kaggle Models](https://www.kaggle.com/models). KerasNLP models will now work in Kaggle offline notebooks and all assets will quickly attach to a notebook rather than needing a slow download.

Summary

KerasNLP pre-trained models are now all made available through [Kaggle Models](https://www.kaggle.com/models). You can see all models currently available in both KerasCV and KerasNLP [here](https://www.kaggle.com/organizations/keras/models). Individual [model pages](https://www.kaggle.com/models/keras/gpt2) will include example usage and a file browser to examine all available assets for a model preset.

This change will not affect the existing usage of `from_preset()`. Statement like `keras_nlp.models.BertClassifier.from_preset("bert_base_en")` will continue to work and download checkpoints from the Kaggle Models hub.

A note on model saving—for saving support across Keras 2 and Keras 3, we recommend using the new Keras saved model format. You can use `model.save('path/to/location.keras')` for a full model and `model.save_weights('path/to/location.weights.h5')` for checkpoints. See the [Keras saving guide](https://keras.io/guides/serialization_and_saving/) for more details.

What's Changed
* Don't export model internals publicly by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1255
* Bump master branch version number to 0.7.0.dev0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1254
* Fix/allow different encoder and decoder feature dimensions in transformer decoder layer by ferraric in https://github.com/keras-team/keras-nlp/pull/1260
* Doc updates to switch branding to Keras 3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1259
* Remove unused TPU testing for backbones by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1266
* Make gelu a function, not a lambda so it can be loaded without safe_mode=False by calvingiles in https://github.com/keras-team/keras-nlp/pull/1262
* Update requirements and install instructions for multi-backend keras by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1257
* Support Keras 3 installation by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1258
* Remove dtensor by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1268
* Add a lora dense layer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1263
* Factor out testing routines for models by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1269
* Convert T5 to Keras 3 by nkovela1 in https://github.com/keras-team/keras-nlp/pull/1274
* Fix missing backticks in DistilBertClassifier docstrings by Philmod in https://github.com/keras-team/keras-nlp/pull/1278
* T5 checkpoint conversion with HF by nkovela1 in https://github.com/keras-team/keras-nlp/pull/1277
* Use gelu_approximate directly in t5 presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1284
* Add preset tests and weights URLs by nkovela1 in https://github.com/keras-team/keras-nlp/pull/1285
* Support loading keras 3 nightly by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1286
* Remove the use of `SentencePieceTrainer` from tests by tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1283
* Fix XLM-RoBERTa detokenize() by abheesht17 in https://github.com/keras-team/keras-nlp/pull/1289
* Correct tie_embedding_weights and add logit checking by nkovela1 in https://github.com/keras-team/keras-nlp/pull/1288
* Add detokenize testing for model tokenizers by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1290
* Fix Whisper by abheesht17 in https://github.com/keras-team/keras-nlp/pull/1287
* Test against Keras 3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1273
* Support TF_USE_LEGACY_KERAS by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1295
* Run workflows with read-only tokens by pnacht in https://github.com/keras-team/keras-nlp/pull/1305
* Update CONTRIBUTING.md by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1310
* Add GitHub Action for Nightly by sampathweb in https://github.com/keras-team/keras-nlp/pull/1309
* Fix the publish to pypi action by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1311
* Fix nightly tf failure by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1316
* Switch deberta to use the "int" dtype by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1315
* Add security policy by pnacht in https://github.com/keras-team/keras-nlp/pull/1319
* Fix missing export for reversible embedding by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1327
* Add `version` API to keras_nlp by grasskin in https://github.com/keras-team/keras-nlp/pull/1324
* Fix Keras 3 version check by sampathweb in https://github.com/keras-team/keras-nlp/pull/1328
* Simplify running KerasNLP with Keras 3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1308
* Fix issues with version by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1332
* Fix typo in whisper presets files by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1337
* `ELECTRA` backbone implementation in keras by pranavvp16 in https://github.com/keras-team/keras-nlp/pull/1291
* Fix t5 tokenizer expected output by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1348
* Add __init__.py for electra by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1352
* Remove lora dense for now by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1359
* Adds Kokoro Build script for Keras-NLP GPU tests by sampathweb in https://github.com/keras-team/keras-nlp/pull/1355
* Fixes GPU Test failures for Keras 3 by sampathweb in https://github.com/keras-team/keras-nlp/pull/1361
* Change Continuous config to also run only large tests by sampathweb in https://github.com/keras-team/keras-nlp/pull/1362
* ElectraTokenizer by pranavvp16 in https://github.com/keras-team/keras-nlp/pull/1357
* Add MistralAI's 7B Transformer as a backbone in KerasNLP Models by tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1314
* changing pooling output by mbrhd in https://github.com/keras-team/keras-nlp/pull/1364
* Add `LlamaBackbone` by shivance in https://github.com/keras-team/keras-nlp/pull/1203
* Align pip_build with keras by sampathweb in https://github.com/keras-team/keras-nlp/pull/1374
* Remove cloudbuild config by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1375
* Fix one last bad preset hash by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1381
* Add a tokenizer for the Mistral backbone by tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1383
* Kaggle Presets by sampathweb in https://github.com/keras-team/keras-nlp/pull/1365
* Fix mistral and electra tokenizer to match kaggle changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1387
* Align requirments with Keras by sampathweb in https://github.com/keras-team/keras-nlp/pull/1386
* Add a preprocessor for the Mistral backbone by tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1385
* Switch to always expect full Kaggle preset handles by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1390

New Contributors
* calvingiles made their first contribution in https://github.com/keras-team/keras-nlp/pull/1262
* tirthasheshpatel made their first contribution in https://github.com/keras-team/keras-nlp/pull/1283
* pnacht made their first contribution in https://github.com/keras-team/keras-nlp/pull/1305
* grasskin made their first contribution in https://github.com/keras-team/keras-nlp/pull/1324
* pranavvp16 made their first contribution in https://github.com/keras-team/keras-nlp/pull/1291
* mbrhd made their first contribution in https://github.com/keras-team/keras-nlp/pull/1364

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.6.4...v0.7.0

Summary
This point release simplifies our support for Keras 3 and Keras 2.
- If Keras 2 is installed, KerasNLP will use Keras 2 and TensorFlow.
- If Keras 3 is installed, KerasNLP will use Keras 3 and run on any backend.

If you have any issue installing KerasNLP, please open [an issue](https://github.com/keras-team/keras-nlp/issues/new/choose).

What's Changed
* 0.6.4 cherry picks by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1350
* Version bump for 0.6.4.dev0 pre-release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1351
* Version bump for 0.6.4 release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1356

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.6.3...v0.6.4

Summary

This release adds support for running KerasNLP against Keras 3. You can try this today by installing `tf-nightly` and `tensorflow-text-nightly`.

shell
pip install keras-nlp
pip uninstall -y tensorflow-text tensorflow keras
pip install tensorflow-text-nightly tf-nightly


Otherwise, this release should be a no-op for all users. No new features, no change in default behavior.

Upcoming changes

After the release of Keras 3, we will drop support for running KerasNLP against the Keras Core package (no more `import keras_core as keras`), in favor of Keras 3. Keras 3 is the long-term replacement for Keras Core.

What's Changed
* Cherry picks for 0.6.3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1297
* Version bump 0.6.3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1298
* Bump the version to 0.6.3.dev1 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1301
* Version bump to 0.6.3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1302


**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.6.2...v0.6.3