Keras-hub

In this 0.5 release, we are bringing generative AI to KerasNLP!

Summary
* Added text generation task model `keras_nlp.models.GPT2CausalLM` and `keras_nlp.models.OPTCausalLM` along with corresponding preprocessors. Both task models exposed a public `generate()` method for text generation.
* Refactored text generation utils into sampler APIs in `keras_nlp.samplers` for better UX and scalability.
* Added MaskedLM task models `keras_nlp.models.XXXMaskedLM`, e.g., `keras_nlp.models.BertMaskedLM`.

What's Changed
* Update python version in readme to 3.8 by haifeng-jin in https://github.com/keras-team/keras-nlp/pull/618
* Modify our pip install line so we upgrade tf by mattdangerw in https://github.com/keras-team/keras-nlp/pull/616
* Use Adam optimizer for quick start by mattdangerw in https://github.com/keras-team/keras-nlp/pull/620
* Clean up class name and `self` in calls to `super()` by mbrukman in https://github.com/keras-team/keras-nlp/pull/628
* Update word_piece_tokenizer.py by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/617
* Add DeBERTaV3 Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/633
* Add AlbertTokenizer and AlbertPreprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/627
* Create `Backbone` base class by jbischof in https://github.com/keras-team/keras-nlp/pull/621
* Add TPU testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/591
* Add Base Preprocessor Class by abheesht17 in https://github.com/keras-team/keras-nlp/pull/638
* Add keras_nlp.samplers by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/563
* Add ALBERT Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/622
* Add a small script to count parameters in our presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/610
* Clean up examples/ directory by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/637
* Fix Small BERT Typo by abheesht17 in https://github.com/keras-team/keras-nlp/pull/651
* Rename examples/bert -> examples/bert_pretraining by mattdangerw in https://github.com/keras-team/keras-nlp/pull/647
* Add FNet Preprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/646
* Add FNet Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/643
* Small DeBERTa Docstring Fixes by abheesht17 in https://github.com/keras-team/keras-nlp/pull/666
* Add Fenced Docstring Testing by abheesht17 in https://github.com/keras-team/keras-nlp/pull/640
* Corrected the epsilon value by soma2000-lang in https://github.com/keras-team/keras-nlp/pull/665
* Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/654
* Fix `value_dim` in `TransformerDecoder`'s cross-attn layer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
* Add ALBERT Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/655
* Add Base Task Class by abheesht17 in https://github.com/keras-team/keras-nlp/pull/671
* Implement TopP, TopK and Beam samplers by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/652
* Add FNet Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/659
* Bump the year to 2023 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/679
* Add BART Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/661
* Handle trainable and name in the backbone base class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/680
* Ignore Task Docstring for Testing by abheesht17 in https://github.com/keras-team/keras-nlp/pull/683
* Light-weight benchmarking script by NusretOzates in https://github.com/keras-team/keras-nlp/pull/664
* Conditionally import tf_text everywhere by mattdangerw in https://github.com/keras-team/keras-nlp/pull/684
* Expose `token_embedding` as a Backbone Property by abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
* Move `from_preset` to base tokenizer classes by shivance in https://github.com/keras-team/keras-nlp/pull/673
* add f_net_classifier and f_net_classifier_test by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/670
* import rouge_scorer directly from rouge_score package by sampathweb in https://github.com/keras-team/keras-nlp/pull/691
* Fix typo in requirements file juypter -> jupyter by mattdangerw in https://github.com/keras-team/keras-nlp/pull/693
* Temporary fix to get nightly green again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/696
* GPT2 Text Generation APIs by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/592
* Run keras saving tests on nightly and fix RobertaClassifier test by mattdangerw in https://github.com/keras-team/keras-nlp/pull/692
* Speed up pip install keras-nlp; simplify deps by mattdangerw in https://github.com/keras-team/keras-nlp/pull/697
* Add `AlbertClassifier` by shivance in https://github.com/keras-team/keras-nlp/pull/668
* Make tokenizer, backbone, preprocessor properties settable on base class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/700
* Update to latest black by mattdangerw in https://github.com/keras-team/keras-nlp/pull/708
* RobertaMaskedLM task and preprocessor by mattdangerw in https://github.com/keras-team/keras-nlp/pull/653
* Default compilation for BERT/RoBERTa classifiers by jbischof in https://github.com/keras-team/keras-nlp/pull/695
* Add start/end token padding to `GPT2Preprocessor` by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
* Don't install tf stable when building our nightly image by mattdangerw in https://github.com/keras-team/keras-nlp/pull/711
* Add OPT Backbone and Tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/699
* Small OPT Doc-string Edits by abheesht17 in https://github.com/keras-team/keras-nlp/pull/716
* Default compilation other classifiers by Plutone11011 in https://github.com/keras-team/keras-nlp/pull/714
* Add BartTokenizer and BART Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/685
* Add an add_prefix_space Arg in BytePairTokenizer by shivance in https://github.com/keras-team/keras-nlp/pull/715
* Opt presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/707
* fix import of tensorflow_text in tf_utils by sampathweb in https://github.com/keras-team/keras-nlp/pull/723
* Check for masked token in roberta tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/742
* Improve test coverage for special tokens in model tokenizers by mattdangerw in https://github.com/keras-team/keras-nlp/pull/743
* Fix the sampler truncation strategy by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/713
* Add ALBERT Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/736
* Add FNet Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/737
* Add BART Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/739
* Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/731
* Improving the layer Description. by Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/734
* Adding ragged support to SinePositionEncoding by apupneja in https://github.com/keras-team/keras-nlp/pull/751
* Fix trailing space by mattdangerw in https://github.com/keras-team/keras-nlp/pull/755
* Adding an AlbertMaskedLM task + Fix Projection layer dimension in MaskedLMHead by shivance in https://github.com/keras-team/keras-nlp/pull/725
* New docstring example for TokenAndPosition Embedding layer. by Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/760
* Add a note for TPU issues for deberta_v3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/758
* Add missing exports to models API by mattdangerw in https://github.com/keras-team/keras-nlp/pull/763
* Autogenerate preset table by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/690
* Version bump to 0.5.0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/767
* Adding a FNetMaskedLM task model and preprocessor by apupneja in https://github.com/keras-team/keras-nlp/pull/740
* Add a DistilBertMaskedLM task model by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/724
* Add cache support to decoding journey by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/745
* Handle [MASK] token in DebertaV3Tokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/759
* Update README for 2.4.1 release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/757
* Fix typo in test docstring by jbischof in https://github.com/keras-team/keras-nlp/pull/791
* Fixed Incorrect Links for FNet and DeBERTaV3 models by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/793
* Patch 1 - doc-string spell fix by atharvapurdue in https://github.com/keras-team/keras-nlp/pull/781
* Don't rely on core keras initializer config details by mattdangerw in https://github.com/keras-team/keras-nlp/pull/802
* Simplify the cache decoding graph by mattdangerw in https://github.com/keras-team/keras-nlp/pull/780
* Fix Fenced Doc-String 782 by atharvapurdue in https://github.com/keras-team/keras-nlp/pull/785
* Solve 721 Deberta masklm model by Plutone11011 in https://github.com/keras-team/keras-nlp/pull/732
* Add from_config to sampler by mattdangerw in https://github.com/keras-team/keras-nlp/pull/803
* BertMaskedLM Task Model and Preprocessor by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/774
* Stop generation once end_token_id is seen by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/769
* Added model card links for all pretrained models. by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/795
* Initial PR demonstrating public API export logic. by fchollet in https://github.com/keras-team/keras-nlp/pull/747
* Add preset for finetuning GPT2 on CNN news by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/807
* Add API exports for metrics documented on keras.io by shivance in https://github.com/keras-team/keras-nlp/pull/816
* Add API exports for samplers documented on keras.io by shivance in https://github.com/keras-team/keras-nlp/pull/815
* Add API exports for models documented on keras.io by shivance in https://github.com/keras-team/keras-nlp/pull/814
* Add API exports for tokenizers documented on keras.io by shivance in https://github.com/keras-team/keras-nlp/pull/817
* Add API exports for layers documented on keras.io by fchollet in https://github.com/keras-team/keras-nlp/pull/811
* Add keras_nlp.utils public API exports. by fchollet in https://github.com/keras-team/keras-nlp/pull/819
* retrained bert_tiny_uncased_en_sst2_training.ipynb by susnato in https://github.com/keras-team/keras-nlp/pull/771
* Temporary solution to avoid recompilation by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/808
* Call super.config() in BartBackbone's get_config() by shivance in https://github.com/keras-team/keras-nlp/pull/818
* Update typo in README.md by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/821
* Add Whisper Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/801
* Added note for tensorflow-text in the CONTRIBUTING guide by jaygala223 in https://github.com/keras-team/keras-nlp/pull/805
* Roadmap update by jaygala223 in https://github.com/keras-team/keras-nlp/pull/800
* Remove API export decorator from base classes by shivance in https://github.com/keras-team/keras-nlp/pull/824
* Move integration tests out of repo sources. by fchollet in https://github.com/keras-team/keras-nlp/pull/826
* Function merge_padding_and_attention_mask does not return an output with the desired shape when both padding and attention masks are given by abodinier in https://github.com/keras-team/keras-nlp/pull/790
* Adding XXBackboneTPUTests by shivance in https://github.com/keras-team/keras-nlp/pull/839
* Add a t5 tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/852
* Add compilation defaults for the BertMaskedLM task model by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/836
* added __init__ file for t5 by Akorex in https://github.com/keras-team/keras-nlp/pull/853
* Modified Docstring for GPT2CasualLM by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/855
* Rework bert docstrings for progressive disclosure of complexity by mattdangerw in https://github.com/keras-team/keras-nlp/pull/843
* Fix "causal" spelling in export decorator by abheesht17 in https://github.com/keras-team/keras-nlp/pull/861
* Default compilation for Albert, Distilbert, Roberta MaskedLM by shivance in https://github.com/keras-team/keras-nlp/pull/833
* Speed up default BERT testing roughly 3x by mattdangerw in https://github.com/keras-team/keras-nlp/pull/859
* Add compilation defaults for the Fnet MaskedLM task model by soma2000-lang in https://github.com/keras-team/keras-nlp/pull/834
* Default compilation for Debertav3MaskedLM model by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/835
* Remove from_preset from fnet tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/865
* Add T5 backbone by fchollet in https://github.com/keras-team/keras-nlp/pull/828
* Speeding the tests for opt by susnato in https://github.com/keras-team/keras-nlp/pull/886
* Move generate compilation to the task model by mattdangerw in https://github.com/keras-team/keras-nlp/pull/804
* Speeding the tests for xlm_roberta by susnato in https://github.com/keras-team/keras-nlp/pull/885
* Rework DistilBERT docstrings for progressive disclosure of complexity. by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/881
* Speeding the tests for T5 by susnato in https://github.com/keras-team/keras-nlp/pull/888
* Rework OPT docstrings for progressive disclosure of complexity. by Warlord-K in https://github.com/keras-team/keras-nlp/pull/893
* Get our fenced docstring tests working again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/895
* Speed up default RoBERTa testing roughly 3x by shivance in https://github.com/keras-team/keras-nlp/pull/897
* Speeding the tests for whisper by susnato in https://github.com/keras-team/keras-nlp/pull/887
* Update BytePairTokenizerCache to have similar dtypes for x and y in self.factors. by Sruinard in https://github.com/keras-team/keras-nlp/pull/871
* Init `_backbone`, `_tokenizer` and `_preprocessor` in Task by jbischof in https://github.com/keras-team/keras-nlp/pull/899
* Rework Whisper docstrings for progressive disclosure of complexity by susnato in https://github.com/keras-team/keras-nlp/pull/903
* Speed up default DeBERTa_v3 testing roughly 3x by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/905
* Rework docstring of XLMRoberta by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/882
* Stripping the MASK token by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/876
* Possible fix for task.summary() by mattdangerw in https://github.com/keras-team/keras-nlp/pull/901
* Speed up default FNet testing speedups. by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/894
* Added TPU test for DebertaV3Backbone by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/924
* Fix failing TPU tests by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/931
* Add model contribution guide by abheesht17 in https://github.com/keras-team/keras-nlp/pull/820
* Resolved roberta_checkpoint by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/874
* GLUE evaluation automation script by susnato in https://github.com/keras-team/keras-nlp/pull/848
* Ensure shape in sample so that the shape is correct after TFLite conversion by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/902
* Returning all Beams and Probs and adding a Testing Unit by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/908
* Roberta docstring reworking by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/910
* Speeding the tests for Albert by soma2000-lang in https://github.com/keras-team/keras-nlp/pull/873
* Mlm mask generator docstring adding example by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/916
* Don't save traces for saved model by mattdangerw in https://github.com/keras-team/keras-nlp/pull/945
* Bump stable tf version to 2.12 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/944
* Speeding the tests for DistilBert by soma2000-lang in https://github.com/keras-team/keras-nlp/pull/872
* Allow BPE to treat special tokens as one token by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/939
* Edit examples in samplers by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/957
* Add RandomSampler to Samplers by abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/952
* Add BartPreprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/856
* Remove the old sampler utilities by mattdangerw in https://github.com/keras-team/keras-nlp/pull/948
* Use direct imports everywhere in library by mattdangerw in https://github.com/keras-team/keras-nlp/pull/961
* Update docstrings for relocated `sampler` arg by jbischof in https://github.com/keras-team/keras-nlp/pull/964
* Fix gpt2, t5 and fnet under mixed precision by mattdangerw in https://github.com/keras-team/keras-nlp/pull/958
* Small fixes for special_tokens arg in BPE by abheesht17 in https://github.com/keras-team/keras-nlp/pull/969
* Add contrastive sampler by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/896
* Mark num_classes as required in Classifier classes by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/971
* Rework model docstrings for progressive disclosure of complexity for f_net by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/879
* Handle OOV token in XLMRoBERTaTokenizer's token_to_id function by abheesht17 in https://github.com/keras-team/keras-nlp/pull/968
* Clean up the docker and lint setup by haifeng-jin in https://github.com/keras-team/keras-nlp/pull/981
* Update generate() to work like fit() and predict() by mattdangerw in https://github.com/keras-team/keras-nlp/pull/932
* Speed top-p sampler up by only sampling from top-k tokens by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/980
* Expose the generate_step compilable function by mattdangerw in https://github.com/keras-team/keras-nlp/pull/982
* Fix decoder inputs in BART preprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/984
* Convert string tensors to python strings in `generate()` by mattdangerw in https://github.com/keras-team/keras-nlp/pull/983
* Adding a temperature argument to the base sampler class and related tests by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/951
* Track the task preprocessor layer as part of model by mattdangerw in https://github.com/keras-team/keras-nlp/pull/985
* Add an XLMRobertaMaskedLM task model by shivance in https://github.com/keras-team/keras-nlp/pull/950
* Add an activation argument to all classifiers by mattdangerw in https://github.com/keras-team/keras-nlp/pull/991
* Remove activation from README quickstart by mattdangerw in https://github.com/keras-team/keras-nlp/pull/992
* Rework albert docstrings by mattdangerw in https://github.com/keras-team/keras-nlp/pull/993
* Rework bart docstrings by mattdangerw in https://github.com/keras-team/keras-nlp/pull/994
* Rework deberta docstrings by mattdangerw in https://github.com/keras-team/keras-nlp/pull/995
* Misc fixes to docstrings by mattdangerw in https://github.com/keras-team/keras-nlp/pull/996
* Added temperature argument to the Contrastive Sampler by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/997
* Add `OPTCausalLM` and preprocessors by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/990
* Version bump to 0.5.0.dev0 by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1002
* Add a flag to restrict which docstring tests run by mattdangerw in https://github.com/keras-team/keras-nlp/pull/999
* fix docstring for 0.5 release by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1005
* Serialize activation fn properly by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1007
* Try adding an error if activation and loss are mismatched by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1008
* Fix docstring for 0.5 release by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1009
* Switch to using pip_build for release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1011
* Make version number SSoT. by fchollet in https://github.com/keras-team/keras-nlp/pull/827
* Add DTensor layout map class method for OPT by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1000
* Add DTensor layout map class method for GPT-2 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1014
* Standalone functions for generate pre/post processing for GPT-2 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/998
* install namex in the publish workflow by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1020
* Update publish-to-pypi.yml by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1021
* Standalone functions for generate pre/post processing for OPT by mattdangerw in https://github.com/keras-team/keras-nlp/pull/1015

New Contributors
* haifeng-jin made their first contribution in https://github.com/keras-team/keras-nlp/pull/618
* mbrukman made their first contribution in https://github.com/keras-team/keras-nlp/pull/628
* soma2000-lang made their first contribution in https://github.com/keras-team/keras-nlp/pull/665
* NusretOzates made their first contribution in https://github.com/keras-team/keras-nlp/pull/664
* shivance made their first contribution in https://github.com/keras-team/keras-nlp/pull/673
* Plutone11011 made their first contribution in https://github.com/keras-team/keras-nlp/pull/714
* TheAthleticCoder made their first contribution in https://github.com/keras-team/keras-nlp/pull/731
* Neeshamraghav012 made their first contribution in https://github.com/keras-team/keras-nlp/pull/734
* apupneja made their first contribution in https://github.com/keras-team/keras-nlp/pull/751
* Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/690
* atharvapurdue made their first contribution in https://github.com/keras-team/keras-nlp/pull/781
* fchollet made their first contribution in https://github.com/keras-team/keras-nlp/pull/747
* susnato made their first contribution in https://github.com/keras-team/keras-nlp/pull/771
* jaygala223 made their first contribution in https://github.com/keras-team/keras-nlp/pull/805
* abodinier made their first contribution in https://github.com/keras-team/keras-nlp/pull/790
* Akorex made their first contribution in https://github.com/keras-team/keras-nlp/pull/853
* Warlord-K made their first contribution in https://github.com/keras-team/keras-nlp/pull/893
* Sruinard made their first contribution in https://github.com/keras-team/keras-nlp/pull/871
* abuelnasr0 made their first contribution in https://github.com/keras-team/keras-nlp/pull/882

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.5.0

The 0.4.1 release is a minor release with new model architectures and compilation defaults for task models. If you encounter any problems or have questions, please open an issue!

Summary
* Added compilation defaults for all **task** models (e.g. `keras_nlp.models.BertClassifier`). No existing functionality is changed, but users of task models can now skip calling `.compile()` and use default learning rates and optimization strategies provided by the library.
* Added `keras_nlp.models.AlbertBackbone`, `keras_nlp.models.AlbertClassifier`, preprocessor, and tokenizer layers for pre-trained [ALBERT](https://arxiv.org/abs/1909.11942) models.
* Added `keras_nlp.models.FNetBackbone`, `keras_nlp.models.FNetClassifier`, preprocessor, and tokenizer layers for pre-trained [FNet](https://arxiv.org/abs/2105.03824) models.
* Added `keras_nlp.models.DebertaV3Backbone`, `keras_nlp.models.DebertaV3Classifier`, preprocessor, and tokenizer layers for pre-trained [DeBERTaV3](https://arxiv.org/abs/2111.09543) models.

What's Changed
* Update python version in readme to 3.8 by haifeng-jin in https://github.com/keras-team/keras-nlp/pull/618
* Modify our pip install line so we upgrade tf by mattdangerw in https://github.com/keras-team/keras-nlp/pull/616
* Use Adam optimizer for quick start by mattdangerw in https://github.com/keras-team/keras-nlp/pull/620
* Clean up class name and `self` in calls to `super()` by mbrukman in https://github.com/keras-team/keras-nlp/pull/628
* Update word_piece_tokenizer.py by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/617
* Add DeBERTaV3 Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/633
* Add AlbertTokenizer and AlbertPreprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/627
* Create `Backbone` base class by jbischof in https://github.com/keras-team/keras-nlp/pull/621
* Add TPU testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/591
* Add Base Preprocessor Class by abheesht17 in https://github.com/keras-team/keras-nlp/pull/638
* Add keras_nlp.samplers by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/563
* Add ALBERT Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/622
* Add a small script to count parameters in our presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/610
* Clean up examples/ directory by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/637
* Fix Small BERT Typo by abheesht17 in https://github.com/keras-team/keras-nlp/pull/651
* Rename examples/bert -> examples/bert_pretraining by mattdangerw in https://github.com/keras-team/keras-nlp/pull/647
* Add FNet Preprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/646
* Add FNet Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/643
* Small DeBERTa Docstring Fixes by abheesht17 in https://github.com/keras-team/keras-nlp/pull/666
* Add Fenced Docstring Testing by abheesht17 in https://github.com/keras-team/keras-nlp/pull/640
* Corrected the epsilon value by soma2000-lang in https://github.com/keras-team/keras-nlp/pull/665
* Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/654
* Fix `value_dim` in `TransformerDecoder`'s cross-attn layer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
* Add ALBERT Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/655
* Add Base Task Class by abheesht17 in https://github.com/keras-team/keras-nlp/pull/671
* Implement TopP, TopK and Beam samplers by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/652
* Add FNet Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/659
* Bump the year to 2023 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/679
* Add BART Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/661
* Handle trainable and name in the backbone base class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/680
* Ignore Task Docstring for Testing by abheesht17 in https://github.com/keras-team/keras-nlp/pull/683
* Light-weight benchmarking script by NusretOzates in https://github.com/keras-team/keras-nlp/pull/664
* Conditionally import tf_text everywhere by mattdangerw in https://github.com/keras-team/keras-nlp/pull/684
* Expose `token_embedding` as a Backbone Property by abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
* Move `from_preset` to base tokenizer classes by shivance in https://github.com/keras-team/keras-nlp/pull/673
* add f_net_classifier and f_net_classifier_test by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/670
* import rouge_scorer directly from rouge_score package by sampathweb in https://github.com/keras-team/keras-nlp/pull/691
* Fix typo in requirements file juypter -> jupyter by mattdangerw in https://github.com/keras-team/keras-nlp/pull/693
* Temporary fix to get nightly green again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/696
* GPT2 Text Generation APIs by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/592
* Run keras saving tests on nightly and fix RobertaClassifier test by mattdangerw in https://github.com/keras-team/keras-nlp/pull/692
* Speed up pip install keras-nlp; simplify deps by mattdangerw in https://github.com/keras-team/keras-nlp/pull/697
* Add `AlbertClassifier` by shivance in https://github.com/keras-team/keras-nlp/pull/668
* Make tokenizer, backbone, preprocessor properties settable on base class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/700
* Update to latest black by mattdangerw in https://github.com/keras-team/keras-nlp/pull/708
* RobertaMaskedLM task and preprocessor by mattdangerw in https://github.com/keras-team/keras-nlp/pull/653
* Default compilation for BERT/RoBERTa classifiers by jbischof in https://github.com/keras-team/keras-nlp/pull/695
* Add start/end token padding to `GPT2Preprocessor` by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
* Don't install tf stable when building our nightly image by mattdangerw in https://github.com/keras-team/keras-nlp/pull/711
* Add OPT Backbone and Tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/699
* Small OPT Doc-string Edits by abheesht17 in https://github.com/keras-team/keras-nlp/pull/716
* Default compilation other classifiers by Plutone11011 in https://github.com/keras-team/keras-nlp/pull/714
* Add BartTokenizer and BART Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/685
* Add an add_prefix_space Arg in BytePairTokenizer by shivance in https://github.com/keras-team/keras-nlp/pull/715
* Opt presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/707
* fix import of tensorflow_text in tf_utils by sampathweb in https://github.com/keras-team/keras-nlp/pull/723
* Check for masked token in roberta tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/742
* Improve test coverage for special tokens in model tokenizers by mattdangerw in https://github.com/keras-team/keras-nlp/pull/743
* Fix the sampler truncation strategy by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/713
* Add ALBERT Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/736
* Add FNet Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/737
* Add BART Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/739
* Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/731
* Improving the layer Description. by Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/734
* Adding ragged support to SinePositionEncoding by apupneja in https://github.com/keras-team/keras-nlp/pull/751
* Fix trailing space by mattdangerw in https://github.com/keras-team/keras-nlp/pull/755
* Adding an AlbertMaskedLM task model and preprocessor by shivance in https://github.com/keras-team/keras-nlp/pull/725
* New docstring example for TokenAndPosition Embedding layer. by Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/760
* Add a note for TPU issues for deberta_v3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/758
* Add missing exports to models API by mattdangerw in https://github.com/keras-team/keras-nlp/pull/763
* Autogenerate preset table by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/690
* Remove work in progress API for 0.4.1 release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/765
* Version bump to 0.4.1.dev0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/766
* Version bump to 0.4.1 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/768

New Contributors
* haifeng-jin made their first contribution in https://github.com/keras-team/keras-nlp/pull/618
* mbrukman made their first contribution in https://github.com/keras-team/keras-nlp/pull/628
* soma2000-lang made their first contribution in https://github.com/keras-team/keras-nlp/pull/665
* NusretOzates made their first contribution in https://github.com/keras-team/keras-nlp/pull/664
* shivance made their first contribution in https://github.com/keras-team/keras-nlp/pull/673
* Plutone11011 made their first contribution in https://github.com/keras-team/keras-nlp/pull/714
* TheAthleticCoder made their first contribution in https://github.com/keras-team/keras-nlp/pull/731
* Neeshamraghav012 made their first contribution in https://github.com/keras-team/keras-nlp/pull/734
* Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/690

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.4.1

Summary
* Dev release to test out the upcoming 0.4.1.

What's Changed
* Update python version in readme to 3.8 by haifeng-jin in https://github.com/keras-team/keras-nlp/pull/618
* Modify our pip install line so we upgrade tf by mattdangerw in https://github.com/keras-team/keras-nlp/pull/616
* Use Adam optimizer for quick start by mattdangerw in https://github.com/keras-team/keras-nlp/pull/620
* Clean up class name and `self` in calls to `super()` by mbrukman in https://github.com/keras-team/keras-nlp/pull/628
* Update word_piece_tokenizer.py by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/617
* Add DeBERTaV3 Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/633
* Add AlbertTokenizer and AlbertPreprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/627
* Create `Backbone` base class by jbischof in https://github.com/keras-team/keras-nlp/pull/621
* Add TPU testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/591
* Add Base Preprocessor Class by abheesht17 in https://github.com/keras-team/keras-nlp/pull/638
* Add keras_nlp.samplers by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/563
* Add ALBERT Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/622
* Add a small script to count parameters in our presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/610
* Clean up examples/ directory by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/637
* Fix Small BERT Typo by abheesht17 in https://github.com/keras-team/keras-nlp/pull/651
* Rename examples/bert -> examples/bert_pretraining by mattdangerw in https://github.com/keras-team/keras-nlp/pull/647
* Add FNet Preprocessor by abheesht17 in https://github.com/keras-team/keras-nlp/pull/646
* Add FNet Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/643
* Small DeBERTa Docstring Fixes by abheesht17 in https://github.com/keras-team/keras-nlp/pull/666
* Add Fenced Docstring Testing by abheesht17 in https://github.com/keras-team/keras-nlp/pull/640
* Corrected the epsilon value by soma2000-lang in https://github.com/keras-team/keras-nlp/pull/665
* Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/654
* Fix `value_dim` in `TransformerDecoder`'s cross-attn layer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
* Add ALBERT Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/655
* Add Base Task Class by abheesht17 in https://github.com/keras-team/keras-nlp/pull/671
* Implement TopP, TopK and Beam samplers by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/652
* Add FNet Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/659
* Bump the year to 2023 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/679
* Add BART Backbone by abheesht17 in https://github.com/keras-team/keras-nlp/pull/661
* Handle trainable and name in the backbone base class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/680
* Ignore Task Docstring for Testing by abheesht17 in https://github.com/keras-team/keras-nlp/pull/683
* Light-weight benchmarking script by NusretOzates in https://github.com/keras-team/keras-nlp/pull/664
* Conditionally import tf_text everywhere by mattdangerw in https://github.com/keras-team/keras-nlp/pull/684
* Expose `token_embedding` as a Backbone Property by abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
* Move `from_preset` to base tokenizer classes by shivance in https://github.com/keras-team/keras-nlp/pull/673
* add f_net_classifier and f_net_classifier_test by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/670
* import rouge_scorer directly from rouge_score package by sampathweb in https://github.com/keras-team/keras-nlp/pull/691
* Fix typo in requirements file juypter -> jupyter by mattdangerw in https://github.com/keras-team/keras-nlp/pull/693
* Temporary fix to get nightly green again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/696
* GPT2 Text Generation APIs by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/592
* Run keras saving tests on nightly and fix RobertaClassifier test by mattdangerw in https://github.com/keras-team/keras-nlp/pull/692
* Speed up pip install keras-nlp; simplify deps by mattdangerw in https://github.com/keras-team/keras-nlp/pull/697
* Add `AlbertClassifier` by shivance in https://github.com/keras-team/keras-nlp/pull/668
* Make tokenizer, backbone, preprocessor properties settable on base class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/700
* Update to latest black by mattdangerw in https://github.com/keras-team/keras-nlp/pull/708
* RobertaMaskedLM task and preprocessor by mattdangerw in https://github.com/keras-team/keras-nlp/pull/653
* Default compilation for BERT/RoBERTa classifiers by jbischof in https://github.com/keras-team/keras-nlp/pull/695
* Add start/end token padding to `GPT2Preprocessor` by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
* Don't install tf stable when building our nightly image by mattdangerw in https://github.com/keras-team/keras-nlp/pull/711
* Add OPT Backbone and Tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/699
* Small OPT Doc-string Edits by abheesht17 in https://github.com/keras-team/keras-nlp/pull/716
* Default compilation other classifiers by Plutone11011 in https://github.com/keras-team/keras-nlp/pull/714
* Add BartTokenizer and BART Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/685
* Add an add_prefix_space Arg in BytePairTokenizer by shivance in https://github.com/keras-team/keras-nlp/pull/715
* Opt presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/707
* fix import of tensorflow_text in tf_utils by sampathweb in https://github.com/keras-team/keras-nlp/pull/723
* Check for masked token in roberta tokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/742
* Improve test coverage for special tokens in model tokenizers by mattdangerw in https://github.com/keras-team/keras-nlp/pull/743
* Fix the sampler truncation strategy by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/713
* Add ALBERT Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/736
* Add FNet Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/737
* Add BART Conversion Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/739
* Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/731
* Improving the layer Description. by Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/734
* Adding ragged support to SinePositionEncoding by apupneja in https://github.com/keras-team/keras-nlp/pull/751
* Fix trailing space by mattdangerw in https://github.com/keras-team/keras-nlp/pull/755
* Adding an AlbertMaskedLM task model and preprocessor by shivance in https://github.com/keras-team/keras-nlp/pull/725
* New docstring example for TokenAndPosition Embedding layer. by Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/760
* Add a note for TPU issues for deberta_v3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/758
* Add missing exports to models API by mattdangerw in https://github.com/keras-team/keras-nlp/pull/763
* Autogenerate preset table by Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/690
* Version bump to 0.5.0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/767

New Contributors
* haifeng-jin made their first contribution in https://github.com/keras-team/keras-nlp/pull/618
* mbrukman made their first contribution in https://github.com/keras-team/keras-nlp/pull/628
* soma2000-lang made their first contribution in https://github.com/keras-team/keras-nlp/pull/665
* NusretOzates made their first contribution in https://github.com/keras-team/keras-nlp/pull/664
* shivance made their first contribution in https://github.com/keras-team/keras-nlp/pull/673
* Plutone11011 made their first contribution in https://github.com/keras-team/keras-nlp/pull/714
* TheAthleticCoder made their first contribution in https://github.com/keras-team/keras-nlp/pull/731
* Neeshamraghav012 made their first contribution in https://github.com/keras-team/keras-nlp/pull/734
* apupneja made their first contribution in https://github.com/keras-team/keras-nlp/pull/751
* Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/690

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.4.1.dev0

The 0.4 release adds support for pretrained models to the library via `keras_nlp.models`. You can read an
introduction to the new API in our [Getting Started Guide](https://keras.io/guides/keras_nlp/getting_started/).

If you encounter any problems or have questions, please open an issue!

Breaking Changes
* Renamed `keras_nlp.layers.MLMHead` -> `keras_nlp.layers.MaskedLMHead`.
* Renamed `keras_nlp.layers.MLMMaskGenerator` -> `keras_nlp.layers.MaskedLMMaskGenerator`.
* Renamed `keras_nlp.layers.UnicodeCharacterTokenizer` -> `keras_nlp.layers.UnicodeCodepointTokenizer`.
* Switched the default of `lowercase` in `keras_nlp.tokenizers.WordPieceTokenizer` from `True` to `False`.
* Renamed the token id output of `MaskedLMMaskGenerator` from `"tokens"` to `"tokens_ids"`.

Summary
* Added the `keras_nlp.models` API.
* Added support for BERT, DistilBERT, RoBERTa, and XLM-RoBERTa models and pretrained checkpoints.
* See our [Getting Started Guide](https://keras.io/guides/keras_nlp/getting_started/) for more details.
* Added new metrics.
* `keras_nlp.metrics.Bleu` and `keras_nlp.metrics.EditDistance`.
* Added new vocabulary training utilities.
* `keras_nlp.tokenizers.compute_word_piece_vocabulary` and `keras_nlp.tokenizers.compute_sentence_piece_proto`.
* Added new preprocessing layers.
* `keras_nlp.layers.RandomSwap` and `keras_nlp.layers.RandomDeletion`.

What's Changed
* Add Edit Distance Metric by abheesht17 in https://github.com/keras-team/keras-nlp/pull/231
* Minor fix to simplify and test handling of max_length prompts by jbischof in https://github.com/keras-team/keras-nlp/pull/258
* Remove split regex args for WordPieceTokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/255
* Add instructions on installing the latest changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/261
* Add warning when k > vocab_size in top_k_search by jbischof in https://github.com/keras-team/keras-nlp/pull/260
* Fix keras library imports and usage by jbischof in https://github.com/keras-team/keras-nlp/pull/262
* Add BLEU Score by abheesht17 in https://github.com/keras-team/keras-nlp/pull/222
* Configure GKE-based accelerator testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/265
* Added WordPieceTokenizer training function by jessechancy in https://github.com/keras-team/keras-nlp/pull/256
* Add requirements.txt for cloud build by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/267
* Global Seed Bug Fix by jessechancy in https://github.com/keras-team/keras-nlp/pull/269
* Update accelerator testing to use the new GCP project by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/272
* Fixed typo: "recieved" by ehrencrona in https://github.com/keras-team/keras-nlp/pull/273
* Reuse dense pooled output for fine tuning by mattdangerw in https://github.com/keras-team/keras-nlp/pull/251
* Simplify BERT modeling, use keras embeddings by mattdangerw in https://github.com/keras-team/keras-nlp/pull/253
* Rename UnicodeCharacterTokenizer>UnicodeCodepointTokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/254
* Add README for accelerator testing config folder by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/276
* Random Deletion Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/214
* Made trainer more efficient. Loading full files instead of using TextLineDataset. by jessechancy in https://github.com/keras-team/keras-nlp/pull/280
* Use KerasNLP for BERT preprocessing for GLUE by mattdangerw in https://github.com/keras-team/keras-nlp/pull/252
* Minor fixes to the Random Deletion Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/286
* Fixes for WordPieceTrainer by aflah02 in https://github.com/keras-team/keras-nlp/pull/293
* Update default to strip_accents=False by jessechancy in https://github.com/keras-team/keras-nlp/pull/289
* Move Bert to models folder by jbischof in https://github.com/keras-team/keras-nlp/pull/288
* Make Decoding Functions Graph-compatible (with XLA Support!) by abheesht17 in https://github.com/keras-team/keras-nlp/pull/271
* SentencePieceTrainer by aflah02 in https://github.com/keras-team/keras-nlp/pull/281
* Rename `models.Bert()` to `models.BertCustom()` by jbischof in https://github.com/keras-team/keras-nlp/pull/310
* Add a test for variable sequence length inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/313
* Support checkpoint loading for `BertBase` by jbischof in https://github.com/keras-team/keras-nlp/pull/299
* RoBERTa pretrained model forward pass by jessechancy in https://github.com/keras-team/keras-nlp/pull/304
* Register objects as serializable by mattdangerw in https://github.com/keras-team/keras-nlp/pull/292
* Style merging for Bert and Roberta by jbischof in https://github.com/keras-team/keras-nlp/pull/315
* Streamline and speed up tests by jbischof in https://github.com/keras-team/keras-nlp/pull/324
* Add Support for CJK Char Splitting for WordPiece Tokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/318
* Clean up model input names for consistency by mattdangerw in https://github.com/keras-team/keras-nlp/pull/327
* Return a single tensor from roberta by mattdangerw in https://github.com/keras-team/keras-nlp/pull/328
* BERT, RoBERTa: Add `model.compile` UTs by abheesht17 in https://github.com/keras-team/keras-nlp/pull/330
* Continue rename of bert model inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/329
* Text Generation Utilities: Add Support for Ragged Inputs by abheesht17 in https://github.com/keras-team/keras-nlp/pull/300
* `bert_base_zh`, `bert_base_multi_cased`: Add BERT Base Variants by abheesht17 in https://github.com/keras-team/keras-nlp/pull/319
* WordPiece vocabularies trainer on Wikipedia dataset by jessechancy in https://github.com/keras-team/keras-nlp/pull/316
* Use the exported ragged ops for RandomDeletion by mattdangerw in https://github.com/keras-team/keras-nlp/pull/332
* Random Swap Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/224
* Fixes for Random Deletion Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/339
* Move cloudbuild to a hidden directory by mattdangerw in https://github.com/keras-team/keras-nlp/pull/345
* Fix the build by mattdangerw in https://github.com/keras-team/keras-nlp/pull/349
* Migrating from Datasets to TFDS for GLUE Example by aflah02 in https://github.com/keras-team/keras-nlp/pull/340
* Move network_tests into keras_nlp/ by mattdangerw in https://github.com/keras-team/keras-nlp/pull/344
* Stop hardcoding 2.9 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/351
* Add BERT Large by abheesht17 in https://github.com/keras-team/keras-nlp/pull/331
* Add normalize_first arg to Transformer Layers by abheesht17 in https://github.com/keras-team/keras-nlp/pull/350
* Add Small BERT Variants by abheesht17 in https://github.com/keras-team/keras-nlp/pull/338
* Beam Search: Add Ragged and XLA Support by abheesht17 in https://github.com/keras-team/keras-nlp/pull/341
* Fix download paths for bert weights by mattdangerw in https://github.com/keras-team/keras-nlp/pull/356
* Add a BertPreprocessor class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/343
* Text Generation Functions: Add Benchmark Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/342
* Improve readability for encoder/decoder blocks by mattdangerw in https://github.com/keras-team/keras-nlp/pull/353
* Add GPT-2 Model and its Variants by abheesht17 in https://github.com/keras-team/keras-nlp/pull/354
* Clean up BERT, RoBERTa doc-strings by abheesht17 in https://github.com/keras-team/keras-nlp/pull/359
* Create unique string id for each BERT backbone by jbischof in https://github.com/keras-team/keras-nlp/pull/361
* Use model.fit() for BERT Example by abheesht17 in https://github.com/keras-team/keras-nlp/pull/360
* Minor Fixes in BertPreprocessor Layer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/373
* Clone user passed initializers called multiple times by mattdangerw in https://github.com/keras-team/keras-nlp/pull/371
* Update BERT model file structure by mattdangerw in https://github.com/keras-team/keras-nlp/pull/376
* Move gpt model code into a directory by mattdangerw in https://github.com/keras-team/keras-nlp/pull/379
* Move roberta model code into a directory by mattdangerw in https://github.com/keras-team/keras-nlp/pull/380
* Reorg test directories by mattdangerw in https://github.com/keras-team/keras-nlp/pull/384
* Add XLM-RoBERTa by abheesht17 in https://github.com/keras-team/keras-nlp/pull/372
* Add DistilBERT by abheesht17 in https://github.com/keras-team/keras-nlp/pull/382
* Stop running CI on Windows by mattdangerw in https://github.com/keras-team/keras-nlp/pull/386
* Fix Bert serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/385
* Improve MacOS support and pin tensorflow version during testing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/383
* Unify BERT model API in one class by jbischof in https://github.com/keras-team/keras-nlp/pull/387
* Add `from_preset` constructor to `BertPreprocessor` by jbischof in https://github.com/keras-team/keras-nlp/pull/390
* More robustly test BERT preprocessing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/394
* Move `name` and `trainable` to `kwargs` by jbischof in https://github.com/keras-team/keras-nlp/pull/399
* Add `backbone` as `property` for task models by jbischof in https://github.com/keras-team/keras-nlp/pull/398
* Set default name of `Bert` instance to `"backbone"` by jbischof in https://github.com/keras-team/keras-nlp/pull/397
* Fix gpt2 serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/391
* Fix distilbert serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/392
* Fix roberta and xlm-roberta serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/393
* Register the BertPreprocessor as serializable by mattdangerw in https://github.com/keras-team/keras-nlp/pull/401
* BPE tokenizer by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/389
* Change GPT-2's Format to Mirror BERT's by abheesht17 in https://github.com/keras-team/keras-nlp/pull/418
* Fix bert preprocessing docstring so it is runnable by mattdangerw in https://github.com/keras-team/keras-nlp/pull/421
* Change RoBERTa and XLM-RoBERTa's Format to Mirror BERT's by abheesht17 in https://github.com/keras-team/keras-nlp/pull/417
* Update distilbert to mirror recent bert changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/406
* Change gpt2 to GPT2 by sampathweb in https://github.com/keras-team/keras-nlp/pull/425
* Fix byte pair detokenization of 2d arrays by mattdangerw in https://github.com/keras-team/keras-nlp/pull/423
* Never pass Raggeds to user function when generating text by mattdangerw in https://github.com/keras-team/keras-nlp/pull/424
* Add XLMRobertaClassifier by abheesht17 in https://github.com/keras-team/keras-nlp/pull/422
* Add RobertaPreprocessor Layer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/419
* Update Style Guide for naming of Models and Layers by sampathweb in https://github.com/keras-team/keras-nlp/pull/434
* Support String Output for BytePairTokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/438
* Improve our continuous testing for model presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/357
* Remove remote files from BPE docstring by jbischof in https://github.com/keras-team/keras-nlp/pull/440
* Add DistilBertClassifier by abheesht17 in https://github.com/keras-team/keras-nlp/pull/437
* Remove lingering reference to BertCustom by mattdangerw in https://github.com/keras-team/keras-nlp/pull/441
* Add XLM-RoBERTa Tokenizer (SPM) by abheesht17 in https://github.com/keras-team/keras-nlp/pull/428
* Add a disclaimer for use of model checkpoints by mattdangerw in https://github.com/keras-team/keras-nlp/pull/430
* Add a disclaimer to our README by mattdangerw in https://github.com/keras-team/keras-nlp/pull/431
* Fix our BERT GLUE example so it runs again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/444
* Add backbone presets to task classes by jbischof in https://github.com/keras-team/keras-nlp/pull/448
* Split the Bert tokenizer to a separate class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/449
* Conditionally import tf text by mattdangerw in https://github.com/keras-team/keras-nlp/pull/452
* Copy our model disclaimer to the distilbert classifier by mattdangerw in https://github.com/keras-team/keras-nlp/pull/453
* Fix regex string for BPE by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/458
* Fix docstrings and add note to style guide by jbischof in https://github.com/keras-team/keras-nlp/pull/464
* Allow formatting our docstrings inline by mattdangerw in https://github.com/keras-team/keras-nlp/pull/450
* Update self.assertEquals with self.assertEqual by MaximSmolskiy in https://github.com/keras-team/keras-nlp/pull/466
* Document our release process by mattdangerw in https://github.com/keras-team/keras-nlp/pull/473
* Add RobertaTokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/468
* Add DistilBertTokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/469
* Modify XLMRobertaTokenizer to Match BERT by abheesht17 in https://github.com/keras-team/keras-nlp/pull/471
* Add GPT2Tokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/470
* Minor fix to git commands by mattdangerw in https://github.com/keras-team/keras-nlp/pull/475
* Version bump to 0.4.0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/476
* Clarify comment on BERT preset testing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/477
* Fix the nightly build by mattdangerw in https://github.com/keras-team/keras-nlp/pull/484
* Bump tf and tf-text to 2.11 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/490
* Consolidate preset testing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/480
* Allow BertPreprocessor to map labeled datasets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/478
* Glue eval script by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/445
* Update Requirements and Python version in setup.py by sampathweb in https://github.com/keras-team/keras-nlp/pull/495
* First task-level preset with `BertClassifier` by jbischof in https://github.com/keras-team/keras-nlp/pull/494
* Add a helper model to automatically apply preprocessing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/346
* Add GPT2 Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/472
* fix incorrect flag by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/496
* Add instructions on how to update deps of GPU testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/499
* Add XLM-RoBERTa Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/482
* Add DistilBERT Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/479
* Add RoBERTa Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/506
* Fix nightly builds by mattdangerw in https://github.com/keras-team/keras-nlp/pull/522
* Remove typo by jbischof in https://github.com/keras-team/keras-nlp/pull/515
* Fix Model Doc-string Examples by abheesht17 in https://github.com/keras-team/keras-nlp/pull/516
* Mark format.sh executable again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/518
* Make BertClassifier operate directly on raw string inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/485
* Fix nightlies take two by mattdangerw in https://github.com/keras-team/keras-nlp/pull/525
* Use `tf.ones` for docstring example input by jbischof in https://github.com/keras-team/keras-nlp/pull/524
* Fix the index order of GLUE script and other bugs by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/517
* Standardize on "backbone" naming for BERT by jbischof in https://github.com/keras-team/keras-nlp/pull/536
* Add dropout to BertClassifier by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/540
* Preprocess string lists as a batch of single segments by mattdangerw in https://github.com/keras-team/keras-nlp/pull/504
* Update model and file names for DistilBert by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/541
* Rename filenames in models/ to match classnames by mattdangerw in https://github.com/keras-team/keras-nlp/pull/548
* Split tokenizers into their own file by mattdangerw in https://github.com/keras-team/keras-nlp/pull/549
* Add a note about our new file naming conventions by mattdangerw in https://github.com/keras-team/keras-nlp/pull/553
* Add distribution support for GLUE script by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/544
* Make out preset ids more consistent by mattdangerw in https://github.com/keras-team/keras-nlp/pull/552
* Rename DistilBert -> DistilBertBackbone by mattdangerw in https://github.com/keras-team/keras-nlp/pull/551
* Rename Roberta -> RobertaBackbone (and same for XLM*) by mattdangerw in https://github.com/keras-team/keras-nlp/pull/550
* Fix link in glue_benchmark README by mattdangerw in https://github.com/keras-team/keras-nlp/pull/557
* Remove qualification for `PRESET_NAMES` by jbischof in https://github.com/keras-team/keras-nlp/pull/554
* Use `black[jupyter]` to format notebooks by jbischof in https://github.com/keras-team/keras-nlp/pull/556
* Temporarily drop GPT2 from our __init__.py by mattdangerw in https://github.com/keras-team/keras-nlp/pull/560
* Replicate 536 changing GPT2 -> GPT2Backbone by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/558
* Change Backbone Names by abheesht17 in https://github.com/keras-team/keras-nlp/pull/559
* Raise a friendly error message for unbatched input by mattdangerw in https://github.com/keras-team/keras-nlp/pull/545
* Fix Typo in Backbone Doc-strings by abheesht17 in https://github.com/keras-team/keras-nlp/pull/565
* Fix LinearDecayWithWarmup crash in BERT model. by reedwm in https://github.com/keras-team/keras-nlp/pull/564
* Update XLMRobertaPreprocessor to mirror recent changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/568
* Update RobertaPreprocessor to mirror recent changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/567
* Fix minor typo in BertPreprocessor layer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/569
* Update DistilBertPreprocessor to mirror recent changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/566
* Point presets to url containing "v1/" by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/577
* Stop testing h5 saved model format, start testing keras_v3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/521
* fix the gpu testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/581
* Make DistilBertClassifier operate directly on raw string inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/578
* Make RobertaClassifier operate directly on raw string inputs by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/579
* Export XLM-Roberta Classifier by mattdangerw in https://github.com/keras-team/keras-nlp/pull/580
* Make XLMRobertaClassifier operate directly on raw string inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/583
* Fix some misc typos for distilbert by mattdangerw in https://github.com/keras-team/keras-nlp/pull/584
* Report GLUE score and hyperparameter settings by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/585
* Remove support for Python 3.7 to align with keras-nightly by sampathweb in https://github.com/keras-team/keras-nlp/pull/590
* Add DeBERTa v3 Model by abheesht17 in https://github.com/keras-team/keras-nlp/pull/435
* Add DeBERTa Tokenizer and Preprocessor Classes by abheesht17 in https://github.com/keras-team/keras-nlp/pull/589
* Add Dropout to *Classifier Doc-strings by abheesht17 in https://github.com/keras-team/keras-nlp/pull/595
* Rename MLM -> MaskedLM for all library symbols by mattdangerw in https://github.com/keras-team/keras-nlp/pull/598
* File-level Doc-string Changes for Classifiers and Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/604
* Add DebertaClassifier and DeBERTa Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/594
* Rename Deberta -> DebertaV3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/605
* Version bump to 0.4.0.dev0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/609
* Add mixed precision support for glue script by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/608
* Remove deberta from 0.4 release by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/607
* Update README for v0.4 by jbischof in https://github.com/keras-team/keras-nlp/pull/588
* Remove the dev prefix for final release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/613
* Rename preset IDs for consistency by mattdangerw in https://github.com/keras-team/keras-nlp/pull/612

New Contributors
* jbischof made their first contribution in https://github.com/keras-team/keras-nlp/pull/258
* ehrencrona made their first contribution in https://github.com/keras-team/keras-nlp/pull/273
* sampathweb made their first contribution in https://github.com/keras-team/keras-nlp/pull/425
* MaximSmolskiy made their first contribution in https://github.com/keras-team/keras-nlp/pull/466
* ADITYADAS1999 made their first contribution in https://github.com/keras-team/keras-nlp/pull/541
* reedwm made their first contribution in https://github.com/keras-team/keras-nlp/pull/564

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.4.0

**⚠️⚠️⚠️ This is a pre-release for testing purposes, documentation for this release has not yet shipped.**

The KerasNLP 0.4 adds support for pretrained models to the API via `keras_nlp.models`. If you encounter any problems or have questions, please open an issue or discussion of the discussion tab!

Breaking Changes
* Renamed `keras_nlp.layers.MLMHead` -> `keras_nlp.layers.MaskedLMHead`.
* Renamed `keras_nlp.layers.MLMMaskGenerator` -> `keras_nlp.layers.MaskedLMMaskGenerator`.
* Renamed `keras_nlp.layers.UnicodeCharacterTokenizer` -> `keras_nlp.layers.UnicodeCodepointTokenizer`.
* Switched the default of `lowercase` in `keras_nlp.tokenizers.WordPieceTokenizer` from `True` to `False`.
* Renamed the token id output of `MaskedLMMaskGenerator` from `"tokens"` to `"tokens_ids"`.

Summary
* Added the `keras_nlp.models` API.
* Adds support for BERT, DistilBERT, RoBERTa, and XLM-RoBERTa models and pretrained checkpoints.
* Added new metrics.
* `keras_nlp.metrics.Bleu` and `keras_nlp.metrics.EditDistance`.
* Added new vocabulary training utilities.
* `keras_nlp.tokenizers.compute_word_piece_vocabulary` and `keras_nlp.tokenizers.compute_sentence_piece_proto`.

What's Changed
* Add Edit Distance Metric by abheesht17 in https://github.com/keras-team/keras-nlp/pull/231
* Minor fix to simplify and test handling of max_length prompts by jbischof in https://github.com/keras-team/keras-nlp/pull/258
* Remove split regex args for WordPieceTokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/255
* Add instructions on installing the latest changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/261
* Add warning when k > vocab_size in top_k_search by jbischof in https://github.com/keras-team/keras-nlp/pull/260
* Fix keras library imports and usage by jbischof in https://github.com/keras-team/keras-nlp/pull/262
* Add BLEU Score by abheesht17 in https://github.com/keras-team/keras-nlp/pull/222
* Configure GKE-based accelerator testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/265
* Added WordPieceTokenizer training function by jessechancy in https://github.com/keras-team/keras-nlp/pull/256
* Add requirements.txt for cloud build by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/267
* Global Seed Bug Fix by jessechancy in https://github.com/keras-team/keras-nlp/pull/269
* Update accelerator testing to use the new GCP project by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/272
* Fixed typo: "recieved" by ehrencrona in https://github.com/keras-team/keras-nlp/pull/273
* Reuse dense pooled output for fine tuning by mattdangerw in https://github.com/keras-team/keras-nlp/pull/251
* Simplify BERT modeling, use keras embeddings by mattdangerw in https://github.com/keras-team/keras-nlp/pull/253
* Rename UnicodeCharacterTokenizer>UnicodeCodepointTokenizer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/254
* Add README for accelerator testing config folder by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/276
* Random Deletion Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/214
* Made trainer more efficient. Loading full files instead of using TextLineDataset. by jessechancy in https://github.com/keras-team/keras-nlp/pull/280
* Use KerasNLP for BERT preprocessing for GLUE by mattdangerw in https://github.com/keras-team/keras-nlp/pull/252
* Minor fixes to the Random Deletion Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/286
* Fixes for WordPieceTrainer by aflah02 in https://github.com/keras-team/keras-nlp/pull/293
* Update default to strip_accents=False by jessechancy in https://github.com/keras-team/keras-nlp/pull/289
* Move Bert to models folder by jbischof in https://github.com/keras-team/keras-nlp/pull/288
* Make Decoding Functions Graph-compatible (with XLA Support!) by abheesht17 in https://github.com/keras-team/keras-nlp/pull/271
* SentencePieceTrainer by aflah02 in https://github.com/keras-team/keras-nlp/pull/281
* Rename `models.Bert()` to `models.BertCustom()` by jbischof in https://github.com/keras-team/keras-nlp/pull/310
* Add a test for variable sequence length inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/313
* Support checkpoint loading for `BertBase` by jbischof in https://github.com/keras-team/keras-nlp/pull/299
* RoBERTa pretrained model forward pass by jessechancy in https://github.com/keras-team/keras-nlp/pull/304
* Register objects as serializable by mattdangerw in https://github.com/keras-team/keras-nlp/pull/292
* Style merging for Bert and Roberta by jbischof in https://github.com/keras-team/keras-nlp/pull/315
* Streamline and speed up tests by jbischof in https://github.com/keras-team/keras-nlp/pull/324
* Add Support for CJK Char Splitting for WordPiece Tokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/318
* Clean up model input names for consistency by mattdangerw in https://github.com/keras-team/keras-nlp/pull/327
* Return a single tensor from roberta by mattdangerw in https://github.com/keras-team/keras-nlp/pull/328
* BERT, RoBERTa: Add `model.compile` UTs by abheesht17 in https://github.com/keras-team/keras-nlp/pull/330
* Continue rename of bert model inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/329
* Text Generation Utilities: Add Support for Ragged Inputs by abheesht17 in https://github.com/keras-team/keras-nlp/pull/300
* `bert_base_zh`, `bert_base_multi_cased`: Add BERT Base Variants by abheesht17 in https://github.com/keras-team/keras-nlp/pull/319
* WordPiece vocabularies trainer on Wikipedia dataset by jessechancy in https://github.com/keras-team/keras-nlp/pull/316
* Use the exported ragged ops for RandomDeletion by mattdangerw in https://github.com/keras-team/keras-nlp/pull/332
* Random Swap Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/224
* Fixes for Random Deletion Layer by aflah02 in https://github.com/keras-team/keras-nlp/pull/339
* Move cloudbuild to a hidden directory by mattdangerw in https://github.com/keras-team/keras-nlp/pull/345
* Fix the build by mattdangerw in https://github.com/keras-team/keras-nlp/pull/349
* Migrating from Datasets to TFDS for GLUE Example by aflah02 in https://github.com/keras-team/keras-nlp/pull/340
* Move network_tests into keras_nlp/ by mattdangerw in https://github.com/keras-team/keras-nlp/pull/344
* Stop hardcoding 2.9 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/351
* Add BERT Large by abheesht17 in https://github.com/keras-team/keras-nlp/pull/331
* Add normalize_first arg to Transformer Layers by abheesht17 in https://github.com/keras-team/keras-nlp/pull/350
* Add Small BERT Variants by abheesht17 in https://github.com/keras-team/keras-nlp/pull/338
* Beam Search: Add Ragged and XLA Support by abheesht17 in https://github.com/keras-team/keras-nlp/pull/341
* Fix download paths for bert weights by mattdangerw in https://github.com/keras-team/keras-nlp/pull/356
* Add a BertPreprocessor class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/343
* Text Generation Functions: Add Benchmark Script by abheesht17 in https://github.com/keras-team/keras-nlp/pull/342
* Improve readability for encoder/decoder blocks by mattdangerw in https://github.com/keras-team/keras-nlp/pull/353
* Add GPT-2 Model and its Variants by abheesht17 in https://github.com/keras-team/keras-nlp/pull/354
* Clean up BERT, RoBERTa doc-strings by abheesht17 in https://github.com/keras-team/keras-nlp/pull/359
* Create unique string id for each BERT backbone by jbischof in https://github.com/keras-team/keras-nlp/pull/361
* Use model.fit() for BERT Example by abheesht17 in https://github.com/keras-team/keras-nlp/pull/360
* Minor Fixes in BertPreprocessor Layer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/373
* Clone user passed initializers called multiple times by mattdangerw in https://github.com/keras-team/keras-nlp/pull/371
* Update BERT model file structure by mattdangerw in https://github.com/keras-team/keras-nlp/pull/376
* Move gpt model code into a directory by mattdangerw in https://github.com/keras-team/keras-nlp/pull/379
* Move roberta model code into a directory by mattdangerw in https://github.com/keras-team/keras-nlp/pull/380
* Reorg test directories by mattdangerw in https://github.com/keras-team/keras-nlp/pull/384
* Add XLM-RoBERTa by abheesht17 in https://github.com/keras-team/keras-nlp/pull/372
* Add DistilBERT by abheesht17 in https://github.com/keras-team/keras-nlp/pull/382
* Stop running CI on Windows by mattdangerw in https://github.com/keras-team/keras-nlp/pull/386
* Fix Bert serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/385
* Improve MacOS support and pin tensorflow version during testing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/383
* Unify BERT model API in one class by jbischof in https://github.com/keras-team/keras-nlp/pull/387
* Add `from_preset` constructor to `BertPreprocessor` by jbischof in https://github.com/keras-team/keras-nlp/pull/390
* More robustly test BERT preprocessing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/394
* Move `name` and `trainable` to `kwargs` by jbischof in https://github.com/keras-team/keras-nlp/pull/399
* Add `backbone` as `property` for task models by jbischof in https://github.com/keras-team/keras-nlp/pull/398
* Set default name of `Bert` instance to `"backbone"` by jbischof in https://github.com/keras-team/keras-nlp/pull/397
* Fix gpt2 serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/391
* Fix distilbert serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/392
* Fix roberta and xlm-roberta serialization by mattdangerw in https://github.com/keras-team/keras-nlp/pull/393
* Register the BertPreprocessor as serializable by mattdangerw in https://github.com/keras-team/keras-nlp/pull/401
* BPE tokenizer by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/389
* Change GPT-2's Format to Mirror BERT's by abheesht17 in https://github.com/keras-team/keras-nlp/pull/418
* Fix bert preprocessing docstring so it is runnable by mattdangerw in https://github.com/keras-team/keras-nlp/pull/421
* Change RoBERTa and XLM-RoBERTa's Format to Mirror BERT's by abheesht17 in https://github.com/keras-team/keras-nlp/pull/417
* Update distilbert to mirror recent bert changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/406
* Change gpt2 to GPT2 by sampathweb in https://github.com/keras-team/keras-nlp/pull/425
* Fix byte pair detokenization of 2d arrays by mattdangerw in https://github.com/keras-team/keras-nlp/pull/423
* Never pass Raggeds to user function when generating text by mattdangerw in https://github.com/keras-team/keras-nlp/pull/424
* Add XLMRobertaClassifier by abheesht17 in https://github.com/keras-team/keras-nlp/pull/422
* Add RobertaPreprocessor Layer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/419
* Update Style Guide for naming of Models and Layers by sampathweb in https://github.com/keras-team/keras-nlp/pull/434
* Support String Output for BytePairTokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/438
* Improve our continuous testing for model presets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/357
* Remove remote files from BPE docstring by jbischof in https://github.com/keras-team/keras-nlp/pull/440
* Add DistilBertClassifier by abheesht17 in https://github.com/keras-team/keras-nlp/pull/437
* Remove lingering reference to BertCustom by mattdangerw in https://github.com/keras-team/keras-nlp/pull/441
* Add XLM-RoBERTa Tokenizer (SPM) by abheesht17 in https://github.com/keras-team/keras-nlp/pull/428
* Add a disclaimer for use of model checkpoints by mattdangerw in https://github.com/keras-team/keras-nlp/pull/430
* Add a disclaimer to our README by mattdangerw in https://github.com/keras-team/keras-nlp/pull/431
* Fix our BERT GLUE example so it runs again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/444
* Add backbone presets to task classes by jbischof in https://github.com/keras-team/keras-nlp/pull/448
* Split the Bert tokenizer to a separate class by mattdangerw in https://github.com/keras-team/keras-nlp/pull/449
* Conditionally import tf text by mattdangerw in https://github.com/keras-team/keras-nlp/pull/452
* Copy our model disclaimer to the distilbert classifier by mattdangerw in https://github.com/keras-team/keras-nlp/pull/453
* Fix regex string for BPE by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/458
* Fix docstrings and add note to style guide by jbischof in https://github.com/keras-team/keras-nlp/pull/464
* Allow formatting our docstrings inline by mattdangerw in https://github.com/keras-team/keras-nlp/pull/450
* Update self.assertEquals with self.assertEqual by MaximSmolskiy in https://github.com/keras-team/keras-nlp/pull/466
* Document our release process by mattdangerw in https://github.com/keras-team/keras-nlp/pull/473
* Add RobertaTokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/468
* Add DistilBertTokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/469
* Modify XLMRobertaTokenizer to Match BERT by abheesht17 in https://github.com/keras-team/keras-nlp/pull/471
* Add GPT2Tokenizer by abheesht17 in https://github.com/keras-team/keras-nlp/pull/470
* Minor fix to git commands by mattdangerw in https://github.com/keras-team/keras-nlp/pull/475
* Version bump to 0.4.0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/476
* Clarify comment on BERT preset testing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/477
* Fix the nightly build by mattdangerw in https://github.com/keras-team/keras-nlp/pull/484
* Bump tf and tf-text to 2.11 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/490
* Consolidate preset testing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/480
* Allow BertPreprocessor to map labeled datasets by mattdangerw in https://github.com/keras-team/keras-nlp/pull/478
* Glue eval script by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/445
* Update Requirements and Python version in setup.py by sampathweb in https://github.com/keras-team/keras-nlp/pull/495
* First task-level preset with `BertClassifier` by jbischof in https://github.com/keras-team/keras-nlp/pull/494
* Add a helper model to automatically apply preprocessing by mattdangerw in https://github.com/keras-team/keras-nlp/pull/346
* Add GPT2 Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/472
* fix incorrect flag by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/496
* Add instructions on how to update deps of GPU testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/499
* Add XLM-RoBERTa Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/482
* Add DistilBERT Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/479
* Add RoBERTa Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/506
* Fix nightly builds by mattdangerw in https://github.com/keras-team/keras-nlp/pull/522
* Remove typo by jbischof in https://github.com/keras-team/keras-nlp/pull/515
* Fix Model Doc-string Examples by abheesht17 in https://github.com/keras-team/keras-nlp/pull/516
* Mark format.sh executable again by mattdangerw in https://github.com/keras-team/keras-nlp/pull/518
* Make BertClassifier operate directly on raw string inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/485
* Fix nightlies take two by mattdangerw in https://github.com/keras-team/keras-nlp/pull/525
* Use `tf.ones` for docstring example input by jbischof in https://github.com/keras-team/keras-nlp/pull/524
* Fix the index order of GLUE script and other bugs by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/517
* Standardize on "backbone" naming for BERT by jbischof in https://github.com/keras-team/keras-nlp/pull/536
* Add dropout to BertClassifier by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/540
* Preprocess string lists as a batch of single segments by mattdangerw in https://github.com/keras-team/keras-nlp/pull/504
* Update model and file names for DistilBert by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/541
* Rename filenames in models/ to match classnames by mattdangerw in https://github.com/keras-team/keras-nlp/pull/548
* Split tokenizers into their own file by mattdangerw in https://github.com/keras-team/keras-nlp/pull/549
* Add a note about our new file naming conventions by mattdangerw in https://github.com/keras-team/keras-nlp/pull/553
* Add distribution support for GLUE script by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/544
* Make out preset ids more consistent by mattdangerw in https://github.com/keras-team/keras-nlp/pull/552
* Rename DistilBert -> DistilBertBackbone by mattdangerw in https://github.com/keras-team/keras-nlp/pull/551
* Rename Roberta -> RobertaBackbone (and same for XLM*) by mattdangerw in https://github.com/keras-team/keras-nlp/pull/550
* Fix link in glue_benchmark README by mattdangerw in https://github.com/keras-team/keras-nlp/pull/557
* Remove qualification for `PRESET_NAMES` by jbischof in https://github.com/keras-team/keras-nlp/pull/554
* Use `black[jupyter]` to format notebooks by jbischof in https://github.com/keras-team/keras-nlp/pull/556
* Temporarily drop GPT2 from our __init__.py by mattdangerw in https://github.com/keras-team/keras-nlp/pull/560
* Replicate 536 changing GPT2 -> GPT2Backbone by ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/558
* Change Backbone Names by abheesht17 in https://github.com/keras-team/keras-nlp/pull/559
* Raise a friendly error message for unbatched input by mattdangerw in https://github.com/keras-team/keras-nlp/pull/545
* Fix Typo in Backbone Doc-strings by abheesht17 in https://github.com/keras-team/keras-nlp/pull/565
* Fix LinearDecayWithWarmup crash in BERT model. by reedwm in https://github.com/keras-team/keras-nlp/pull/564
* Update XLMRobertaPreprocessor to mirror recent changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/568
* Update RobertaPreprocessor to mirror recent changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/567
* Fix minor typo in BertPreprocessor layer by mattdangerw in https://github.com/keras-team/keras-nlp/pull/569
* Update DistilBertPreprocessor to mirror recent changes by mattdangerw in https://github.com/keras-team/keras-nlp/pull/566
* Point presets to url containing "v1/" by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/577
* Stop testing h5 saved model format, start testing keras_v3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/521
* fix the gpu testing by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/581
* Make DistilBertClassifier operate directly on raw string inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/578
* Make RobertaClassifier operate directly on raw string inputs by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/579
* Export XLM-Roberta Classifier by mattdangerw in https://github.com/keras-team/keras-nlp/pull/580
* Make XLMRobertaClassifier operate directly on raw string inputs by mattdangerw in https://github.com/keras-team/keras-nlp/pull/583
* Fix some misc typos for distilbert by mattdangerw in https://github.com/keras-team/keras-nlp/pull/584
* Report GLUE score and hyperparameter settings by chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/585
* Remove support for Python 3.7 to align with keras-nightly by sampathweb in https://github.com/keras-team/keras-nlp/pull/590
* Add DeBERTa v3 Model by abheesht17 in https://github.com/keras-team/keras-nlp/pull/435
* Add DeBERTa Tokenizer and Preprocessor Classes by abheesht17 in https://github.com/keras-team/keras-nlp/pull/589
* Add Dropout to *Classifier Doc-strings by abheesht17 in https://github.com/keras-team/keras-nlp/pull/595
* Rename MLM -> MaskedLM for all library symbols by mattdangerw in https://github.com/keras-team/keras-nlp/pull/598
* File-level Doc-string Changes for Classifiers and Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/604
* Add DebertaClassifier and DeBERTa Presets by abheesht17 in https://github.com/keras-team/keras-nlp/pull/594
* Rename Deberta -> DebertaV3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/605
* Version bump to 0.4.0.dev0 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/609

New Contributors
* jbischof made their first contribution in https://github.com/keras-team/keras-nlp/pull/258
* ehrencrona made their first contribution in https://github.com/keras-team/keras-nlp/pull/273
* sampathweb made their first contribution in https://github.com/keras-team/keras-nlp/pull/425
* MaximSmolskiy made their first contribution in https://github.com/keras-team/keras-nlp/pull/466
* ADITYADAS1999 made their first contribution in https://github.com/keras-team/keras-nlp/pull/541
* reedwm made their first contribution in https://github.com/keras-team/keras-nlp/pull/564

**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.4.0.dev0

Summary
* Add `keras_nlp.tokenizers.BytePairTokenizer` with `tf.data` friendly support for the tokenization used by GPT-2, RoBERTa and other models.
* Remove the hard dependency on `tensorflow` and `tensorflow-text` when pip installing on MacOS, to accommodate M1 chips. See [this section](https://github.com/keras-team/keras-nlp/blob/master/CONTRIBUTING.md#macos) of our contributor guide for more information on MacOS development.

What's Changed
* Cherry picks 0.3 by mattdangerw in https://github.com/keras-team/keras-nlp/pull/454
* Bump version for 0.3.1 pre release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/456
* Remove dev prefix for 0.3.1 release by mattdangerw in https://github.com/keras-team/keras-nlp/pull/457


**Full Changelog**: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.3.1