keras-hub

Summary

Bug fix patch release.

• Always run tf preprocessing on CPU.
• Fix running preprocessing outside the main python thread.
• Fix loading classifiers with the "old name" of XXClasssifier as XXTextClassifier.
• Restore support for bytestring to tokenizers and other preprocessing layers as strings.

What's Changed

• Version bump for pre-release by @mattdangerw in https://github.com/keras-team/keras-hub/pull/1842
• V0.15.1.dev1 by @mattdangerw in https://github.com/keras-team/keras-hub/pull/1844
• Version bump for 0.15.1 release by @mattdangerw in https://github.com/keras-team/keras-hub/pull/1845

Full Changelog: https://github.com/keras-team/keras-hub/compare/v0.15.0...v0.15.1

Summary

📢 KerasNLP is becoming KerasHub 📢, read more about it here.

This release contains a number of feature improvements:

• Added int8 quantization support.
  • Use the quantize() method to quantize any model.
  • Llama 2 and Llama 3 pre-quantized presets are available.
• PaliGemmaCausalLM will automatically resize input images during preprocessing.
• Added more converters for hugginface/transformers checkpoints.
  • Gemma 2, PaliGemma, GPT2, Bert, Albert, DistilBert, Bart.
• Class detection for huggingface/transformers checkpoints.
  • Call from_preset() on a base class, and we will find the correct subclass to create.
• Added Vicuna presets.
• Alias Classifier as TextClassifier, BertClassifier as BertTextClassifier.
• Added tokenizer.special_tokens and tokenizer.special_token_ids as convenient properties to view all special tokens on a pretrained tokenizer.

# Quantize an unquantized model.
lm = keras_nlp.models.CausalLM.from_preset(
    "gemma2_instruct_2b_en",
    dtype="bfloat16",
)
lm.quantize("int8")
# Load a pre-quantized model.
lm = keras_nlp.models.CausalLM.from_preset(
    "llama3_instruct_8b_en_int8",
    dtype="bfloat16",
)
# Convert a bert model in the huggingface/transformers format.
classifier = keras_nlp.models.TextClassifier.from_preset(
    "hf://google-bert/bert-base-uncased",
    num_classes=2,
)
# View all special tokens.
print(classifier.preprocessor.tokenizer.special_tokens)
print(classifier.preprocessor.tokenizer.special_token_ids)

Breaking changes

• On all backends, all strings and ragged output will be returned as python strings or python lists respectively.
  • This include preprocessing methods like tokenize() and detokenize().
  • This may break code that depended on tf.Tensor output on the tensorflow backend, but will lead to consistent output on all backends, which we believe will be an overall improvement.
  • Preprocessing layers can still always be included in a tf.data preprocessing pipeline, on any backend.

What's Changed

• Version bump to 0.14.0.dev0 by @grasskin in https://github.com/keras-team/keras-nlp/pull/1675
• Revert "Version bump to 0.14.0.dev0" by @grasskin in https://github.com/keras-team/keras-nlp/pull/1676
• Remove Keras pin, fix tests by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1681
• Add quantization support for Gemma, Gemma2 and PaliGemma by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1670
• add vicuna preset by @sineeli in https://github.com/keras-team/keras-nlp/pull/1672
• Porting Gemma 2 transformers checkpoint by @ariG23498 in https://github.com/keras-team/keras-nlp/pull/1678
• Improve CI speed and resolve issues of run_quantization_check by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1682
• Remove build_from_signature from MHA layers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1687
• Refactoring: in CachedMultiHeadAttention call MHA methods instead of recoding the attention calculation by @apehex in https://github.com/keras-team/keras-nlp/pull/1684
• Porting PaliGemma transformers checkpoint by @ariG23498 in https://github.com/keras-team/keras-nlp/pull/1686
• Allow importing keras_nlp without tensorflow by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1660
• Add flag to gemma conversion script to specify local orbax by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1688
• Fix compatibility for earlier versions of Keras by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1690
• Add a test against keras-nightly by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1693
• Fix dtype bugs in ReversibleEmbedding and LayerNorm by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1692
• Partially revert #1687 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1695
• Fix quantization test for XLNet by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1699
• Add a HF BERT converter, improve safetensor loading by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1694
• Add a subtle fix for gemma 2 conversions by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1701
• One more small Gemma conversion fix by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1702
• Slightly more defensive handling of type for backbone by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1703
• Add support for converting Gemma 2 checkpoints by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1700
• Make it clearer what is running in the github action UI by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1707
• Try upgrading tensorflow pin by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1706
• Bump version to fix query norm in Gemma 2 9b by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1709
• Gemma: Add logit soft-capping to score function. by @RyanMullins in https://github.com/keras-team/keras-nlp/pull/1712
• Version bump HEAD to 0.15 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1713
• Port gpt2 transformers checkpoint by @cosmo3769 in https://github.com/keras-team/keras-nlp/pull/1704
• Add soft capping to reversible embedding layer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1718
• Add presets for gemma 2 2b by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1721
• Utilize to_numpy=True in quantize if available by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1725
• Dynamic int8 quantization for Llama2 and Llama3 by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1720
• Bump the python group with 2 updates by @dependabot in https://github.com/keras-team/keras-nlp/pull/1726
• Shield gemma shortnames by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1731
• Sliding window fixes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1738
• Add int8 models to Llama2 and Llama3 by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1734
• Port distilbert transformer checkpoint by @cosmo3769 in https://github.com/keras-team/keras-nlp/pull/1736
• Add support of kwargs to Backbone.from_preset and fix the dtype forwarding in Task.from_preset by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1742
• Remove src init file contents by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1743
• Remove ROADMAP.md by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1773
• Fix nested list in args on keras.io by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1772
• Remove stale tf only examples by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1771
• Limit the default sequence length to 1024 for all models by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1770
• Consistent preprocessing output on all backends by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1777
• Port albert transformer checkpoint by @cosmo3769 in https://github.com/keras-team/keras-nlp/pull/1767
• Lower the default learning rate for albert by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1786
• Port bart transformer checkpoint by @cosmo3769 in https://github.com/keras-team/keras-nlp/pull/1783
• Add an option to disable default compilation by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1787
• Port mistral transformer checkpoint by @cosmo3769 in https://github.com/keras-team/keras-nlp/pull/1768
• [Bart]Fix missing weight port by @cosmo3769 in https://github.com/keras-team/keras-nlp/pull/1789
• Remove python 3.8 version in setup.py by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1792
• Class detection works for huggingface checkpoints by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1800
• Rename KerasNLP symbols for a multi-modal future by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1803
• Move preprocessing to base classes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1807
• Add add_bos=False, add_eos=False to SentencePieceTokenizer.init() by @briango28 in https://github.com/keras-team/keras-nlp/pull/1811
• Only load a full task config when load_task_extras is passed by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1812
• Add image and audio converter classes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1813
• Simplify registering "built-in" presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1818
• Support image and audio information in task summaries by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1819
• Take two of #1812, simpler classifier head loading by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1823
• Remove preprocessing layers we no longer use by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1824
• Version bump for dev release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1825
• Version bump for dev release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1830
• Version bump for 0.15.0 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1832

New Contributors

• @apehex made their first contribution in https://github.com/keras-team/keras-nlp/pull/1684
• @cosmo3769 made their first contribution in https://github.com/keras-team/keras-nlp/pull/1704

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.14.4...v0.15.0

Summary

• Fix issues with Gemma 2 sliding window.
• Fix TensorFlow backend Gemma 2 generation.

What's Changed

• Sliding window fixes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1738
• version bump by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1740
• version bump by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1741

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.14.3...v0.14.4

Summary

• Short names for shield gemma checkpoints.

keras_nlp.models.GemmaCausalLM.from_preset("shieldgemma_2b_en")

What's Changed

• Version bump dev release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1732
• Version bump for release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1733

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.14.2...v0.14.3

Summary

• Add Gemma 2 2b.
• Fixes for logit softcapping.

What's Changed

• Version bump 0.14.2.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1719
• Bump pypi action version by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1722
• version bump by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1723
• Version bump 0.14.2 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1724

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.14.1...v0.14.2

Summary

• Update Gemma 2 9b to fix minor config error.

What's Changed

• Bump version to fix query norm in Gemma 2 9b by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1709
• Version bump 0.14.1.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1714

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.14.0...v0.14.1

Summary

• Add Gemma 2 model!
• Support loading fine-tuned transformers checkpoints in KerasNLP. Loading Gemma and Llama3 models are supported for now and will convert on the fly.
• KerasNLP no longer supports Keras 2. Read Getting started with Keras for more information on installing Keras 3 and compatibility with different frameworks. We recommend using KerasNLP with TensorFlow 2.16 or later, as TF 2.16 packages Keras 3 by default.

What's Changed

• Fix newline characters for pali_gemma by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1655
• Remove dead code by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1659
• Fix some testing on the latest version of keras by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1663
• Vicuna Models checkpoints transfer script by @sineeli in https://github.com/keras-team/keras-nlp/pull/1657
• Add documented but missing methods for some tokenizers by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1664
• Changed from_preset file downloading to use GFile when able by @VarunS1997 in https://github.com/keras-team/keras-nlp/pull/1665
• Fix gfile downloads by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1666
• More error handling for gfile by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1667
• Update error message by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1668
• Ditch Keras 2 support by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1658
• fix GemmaBackbone.get_layout_map + test by @martin-gorner in https://github.com/keras-team/keras-nlp/pull/1669
• Covert a safetensor checkpoint from Hugging Face hub by @ariG23498 in https://github.com/keras-team/keras-nlp/pull/1662
• Add Gemma 2 model by @grasskin in https://github.com/keras-team/keras-nlp/pull/1673
• Version bump to 0.14.0.dev0 by @grasskin in https://github.com/keras-team/keras-nlp/pull/1677

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.12.1...r0.14

Summary

• ⚠️ PaliGemma includes rescaling by default, so images are expected to be passed in the [0, 255] range. This is a backward incompatible change with the original release. Restore the original behavior as follows:

keras_nlp.models.PaliGemmaBackbone.from_preset(
    "pali_gemma_3b_224",
    include_rescaling=False,
)

• Released the Falcon model.

What's Changed

• Update version to 0.13.0 for the master branch by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1640
• Update llama3 preset versions by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1641
• extra argument in save_to_preset method by @sineeli in https://github.com/keras-team/keras-nlp/pull/1634
• Fix a typo in an error handling message by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1647
• Fix a typo in phi3 metadata by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1646
• Add FalconCausalLM by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1635
• Add include rescaling to the pali gemma backbone by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1650
• PaliGemma docstring fix by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1651
• Version bump for 0.12.0.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1652
• Version bump 0.12.1 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1653

New Contributors

• @sineeli made their first contribution in https://github.com/keras-team/keras-nlp/pull/1634

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.12.0...v0.12.1

Summary

Add PaliGemma, Llama 3, and Phi 3 models.

PaliGemma quickstart, see a complete usage on Kaggle.

pali_gemma_lm = keras_nlp.models.PaliGemmaCausalLM.from_preset(
    "pali_gemma_3b_224"
)
pali_gemma_lm.generate(
    inputs={
        "images": images,
        "prompts": prompts,
    }
)

What's Changed

• Add CodeGemma 1.1 presets by @grasskin in https://github.com/keras-team/keras-nlp/pull/1617
• Fix rope scaling factor by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1605
• Fix the issue of propagating training argument in subclasses by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1623
• Pass kwargs to tokenizer when creating preprocessor by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1632
• Add phi3 by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1597
• Add LLaMA 3 tokenizer and preset by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1584
• Export missing llama 3 symbol by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1633
• PaliGemma by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1636
• Update pali_gemma_presets.py by @divyashreepathihalli in https://github.com/keras-team/keras-nlp/pull/1637
• Update version to 0.13.0 for the master branch by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1640
• Update llama3 preset versions by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1641

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.11.1...v0.12.0

Summary

• Add new Code Gemma 1.1 presets, which improve on Code Gemma performance.

What's Changed

• Add CodeGemma 1.1 presets by @grasskin in https://github.com/keras-team/keras-nlp/pull/1617
• Version bump 0.11.1.dev0 by @grasskin in https://github.com/keras-team/keras-nlp/pull/1618
• Version bump 0.11.1 by @grasskin in https://github.com/keras-team/keras-nlp/pull/1619

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.11.0...v0.11.1

Summary

This release has no major feature updates, but changes the location our source code is help. Source code is split into a src/ and api/ directory with an explicit API surface similar to core Keras.

When adding or removing new API in a PR, use ./shell/api_gen.sh to update the autogenerated api/ files. See our contributing guide.

What's Changed

• Change the order of importing keras by @james77777778 in https://github.com/keras-team/keras-nlp/pull/1596
• Add backend info to HF model card by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1599
• Bump required kagglehub version to 0.2.4 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1600
• Bump bert_tiny_en_uncased_sst2 classifier version by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1602
• Allow a task preprocessor to be an argument in from_preset by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1603
• API Generation by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1608
• Update readme with some recent changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1575
• Bump the python group with 2 updates by @dependabot in https://github.com/keras-team/keras-nlp/pull/1611
• Version bump 0.11.0.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1615
• Unexport models from the 0.11 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1614
• Version bump 0.11.0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1616

New Contributors

• @james77777778 made their first contribution in https://github.com/keras-team/keras-nlp/pull/1596

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.10.0...v0.11.0

Summary

• Added support for Task (CausalLM and Classifier) saving and loading which allows uploading Tasks.
• Added basic Model Card for Hugging Face upload.
• Added support for a positions array in our RotaryEmbedding layer.

What's Changed

• 0.9 is out, nightly should be a preview of 0.10 now by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1570
• Do the reverse embedding in the same dtype as the input embedding by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1548
• Add support for positions array in keras_nlp.layers.RotaryEmbedding layer by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1571
• Support Task Saving/Loading by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1547
• Improve error handling for non-keras model loading attempts by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1577
• Add Model Card for Hugging Face Upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1578
• Add Saving Tests by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1590
• Improve error handling for missing TensorFlow dependency in keras_nlp. by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1585
• Fix Keras import by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1593
• Check kagglehub version before upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1594
• Version bump to 0.10.0.dev0 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1595
• Version bump 0.10.0.dev1 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1601
• Version bump to 0.10.0.dev2 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1604
• Version bump to 0.10.0 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1606

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.3...v0.10.0

Summary

• Added support for Task (CausalLM and Classifier) saving and loading which allows uploading Tasks.
• Added basic Model Card for Hugging Face upload.
• Added support for a positions array in our RotaryEmbedding layer.

What's Changed

• 0.9 is out, nightly should be a preview of 0.10 now by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1570
• Do the reverse embedding in the same dtype as the input embedding by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1548
• Add support for positions array in keras_nlp.layers.RotaryEmbedding layer by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1571
• Support Task Saving/Loading by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1547
• Improve error handling for non-keras model loading attempts by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1577
• Add Model Card for Hugging Face Upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1578
• Add Saving Tests by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1590
• Improve error handling for missing TensorFlow dependency in keras_nlp. by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1585
• Fix Keras import by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1593
• Check kagglehub version before upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1594
• Version bump to 0.10.0.dev0 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1595
• Version bump 0.10.0.dev1 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1601
• Version bump to 0.10.0.dev2 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1604

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.3...v0.10.0.dev2

Summary

• Added support for Task (CausalLM and Classifier) saving and loading which allows uploading Tasks.
• Added basic Model Card for Hugging Face upload.
• Added support for a positions array in our RotaryEmbedding layer.

What's Changed

• 0.9 is out, nightly should be a preview of 0.10 now by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1570
• Do the reverse embedding in the same dtype as the input embedding by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1548
• Add support for positions array in keras_nlp.layers.RotaryEmbedding layer by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1571
• Support Task Saving/Loading by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1547
• Improve error handling for non-keras model loading attempts by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1577
• Add Model Card for Hugging Face Upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1578
• Add Saving Tests by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1590
• Improve error handling for missing TensorFlow dependency in keras_nlp. by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1585
• Fix Keras import by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1593
• Check kagglehub version before upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1594
• Version bump to 0.10.0.dev0 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1595
• Version bump 0.10.0.dev1 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1601

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.3...v0.10.0.dev1

Summary

• Add support for Task (CausalLM and Classifier) saving and loading which allows uploading Tasks.
• Add basic Model Card for Hugging Face upload.
• Add support for a positions array in our RotaryEmbedding layer.

What's Changed

• 0.9 is out, nightly should be a preview of 0.10 now by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1570
• Do the reverse embedding in the same dtype as the input embedding by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1548
• Add support for positions array in keras_nlp.layers.RotaryEmbedding layer by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1571
• Support Task Saving/Loading by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1547
• Improve error handling for non-keras model loading attempts by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1577
• Add Model Card for Hugging Face Upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1578
• Add Saving Tests by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1590
• Improve error handling for missing TensorFlow dependency in keras_nlp. by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1585
• Fix Keras import by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1593
• Check kagglehub version before upload by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1594
• Version bump to 0.10.0.dev0 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1595

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.3...v0.10.0.dev0

Patch release with fixes for Llama and Mistral saving.

What's Changed

• Fix saving bug for untied weights with keras 3.2 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1568
• Version bump for dev release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1569
• Version bump 0.9.3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1572

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.2...v0.9.3

Summary

• Initial release of CodeGemma.
• Bump to a Gemma 1.1 version without download issues on Kaggle.

What's Changed

• Fix print_fn issue in task test by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1563
• Update presets for code gemma by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1564
• version bump 0.9.2.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1565
• Version bump 0.9.2 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1566

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.1...v0.9.2

🚧🚧Pre-release🚧🚧

Summary

• Initial release of CodeGemma.
• Bump to a Gemma 1.1 version without download issues on Kaggle.

What's Changed

• Fix print_fn issue in task test by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1563
• Update presets for code gemma by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1564
• version bump 0.9.2.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1565

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.1...v0.9.2.dev0

Patch fix for bug with stop_token_ids.

What's Changed

• Fix the new stop_token_ids argument by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1558
• Fix tests with the "auto" default for stop token ids by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1559
• Version bump for 0.9.1 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1560

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.9.0...v0.9.1

The 0.9.0 release adds new models, hub integrations, and general usability improvements.

Summary

• Added the Gemma 1.1 release.
• Added the Llama 2, BLOOM and ELECTRA models.
• Expose new base classes. Allow from_preset() on base classes.
  • keras_nlp.models.Backbone
  • keras_nlp.models.Task
  • keras_nlp.models.Classifier
  • keras_nlp.models.CausalLM
  • keras_nlp.models.Seq2SeqLM
  • keras_nlp.models.MaskedLM
• Some initial features for uploading to model hubs.
  • backbone.save_to_preset, tokenizer.save_to_preset, keras_nlp.upload_preset.
  • from_preset and upload_preset now work with the Hugging Face Models Hub.
  • More features (task saving, lora saving), and full documentation coming soon.
• Numerical fixes for the Gemma model at mixed_bfloat16 precision. Thanks unsloth for catching!

# Llama 2. Needs Kaggle consent and login, see https://github.com/Kaggle/kagglehub
causal_lm = keras_nlp.models.LlamaCausalLM.from_preset(
    "llama2_7b_en",
    dtype="bfloat16", # Run at half precision for inference.
)
causal_lm.generate("Keras is a", max_length=128)
# Base class usage.
keras_nlp.models.Classifier.from_preset("bert_base_en", num_classes=2)
keras_nlp.models.Tokenizer.from_preset("gemma_2b_en")
keras_nlp.models.CausalLM.from_preset("gpt2_base_en", dtype="mixed_bfloat16")

What's Changed

• Add dtype arg to Gemma HF conversion script by @nkovela1 in https://github.com/keras-team/keras-nlp/pull/1452
• Fix gemma testing import by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1462
• Add docstring for PyTorch conversion script install instructions by @nkovela1 in https://github.com/keras-team/keras-nlp/pull/1471
• Add an annotation to tests that need kaggle auth by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1470
• Fix Mistral memory consumption with JAX and default dtype bug by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1460
• Bump the master version to 0.9 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1473
• Pin to TF 2.16 RC0 by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1478
• Fix gemma rms_normalization's use of epsilon by @cpsauer in https://github.com/keras-team/keras-nlp/pull/1472
• Add FalconBackbone by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1475
• CI - Add kaggle creds to pull model by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1459
• bug in example for ReversibleEmbedding by @TheCrazyT in https://github.com/keras-team/keras-nlp/pull/1484
• doc fix for constrastive sampler by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1488
• Remove broken link to masking and padding guide by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1487
• Fix a typo in causal_lm_preprocessors by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1489
• Fix dtype accessors of tasks/backbones by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1486
• Auto-labels 'gemma' on 'gemma' issues/PRs. by @shmishra99 in https://github.com/keras-team/keras-nlp/pull/1490
• Add BloomCausalLM by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1467
• Remove the bert jupyter conversion notebooks by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1492
• Add FalconTokenizer by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1485
• Add FalconPreprocessor by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1498
• Rename 176B presets & Add other presets into bloom_presets.py by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1496
• Add bloom presets by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1501
• Create workflow for auto assignment of issues and for stale issues by @sachinprasadhs in https://github.com/keras-team/keras-nlp/pull/1495
• Update requirements to TF 2.16 by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1503
• Expose Task and Backbone by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1506
• Clean up and add our gemma conversion script by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1493
• Don't auto-update JAX GPU by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1507
• Keep rope at float32 precision by @grasskin in https://github.com/keras-team/keras-nlp/pull/1497
• Bump the python group with 2 updates by @dependabot in https://github.com/keras-team/keras-nlp/pull/1509
• Fixes for the LLaMA backbone + add dropout by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1499
• Add LlamaPreprocessor and LlamaCausalLMPreprocessor by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1511
• Always run the rotary embedding layer in float32 by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1508
• CI: Fix psutil - Remove install of Python 3.9 and alias of python3 by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1514
• Update gemma_backbone.py for sharding config. by @qlzh727 in https://github.com/keras-team/keras-nlp/pull/1491
• Docs/modelling layers by @mykolaskrynnyk in https://github.com/keras-team/keras-nlp/pull/1502
• Standardize docstring by @sachinprasadhs in https://github.com/keras-team/keras-nlp/pull/1516
• Support tokenization of special tokens for word_piece_tokenizer by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1397
• Upload Model to Kaggle by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1512
• Add scoring mode to MistralCausalLM by @RyanMullins in https://github.com/keras-team/keras-nlp/pull/1521
• Add Mistral Instruct V0.2 preset by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1520
• Add Tests for Kaggle Upload Validation by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1524
• Add presets for Electra and checkpoint conversion script by @pranavvp16 in https://github.com/keras-team/keras-nlp/pull/1384
• Allow saving / loading from Huggingface Hub preset by @Wauplin in https://github.com/keras-team/keras-nlp/pull/1510
• Stop on multiple end tokens by @grasskin in https://github.com/keras-team/keras-nlp/pull/1518
• Fix doc: mistral_base_en -> mistral_7b_en by @asmith26 in https://github.com/keras-team/keras-nlp/pull/1528
• Add lora example to GemmaCausalLM docstring by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1527
• Add LLaMA Causal LM with 7B presets by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1526
• Add task base classes; support out of tree library extensions by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1517
• Doc fixes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1530
• Run the LLaMA and Mistral RMS Layer Norm in float32 by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1532
• Adds score API to GPT-2 by @RyanMullins in https://github.com/keras-team/keras-nlp/pull/1533
• increase pip timeout to 1000s to avoid connection resets by @sampathweb in https://github.com/keras-team/keras-nlp/pull/1535
• Adds the score API to LlamaCausalLM by @RyanMullins in https://github.com/keras-team/keras-nlp/pull/1534
• Implement compute_output_spec() for tokenizers with vocabulary. by @briango28 in https://github.com/keras-team/keras-nlp/pull/1523
• Remove staggler type annotiations by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1536
• Always run SiLU activation in float32 for LLaMA and Mistral by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1540
• Bump the python group with 2 updates by @dependabot in https://github.com/keras-team/keras-nlp/pull/1538
• Disallow saving to preset from keras 2 by @SamanehSaadat in https://github.com/keras-team/keras-nlp/pull/1545
• Fix the rotary embedding computation in LLaMA by @tirthasheshpatel in https://github.com/keras-team/keras-nlp/pull/1544
• Fix re-compilation bugs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1541
• Fix preprocessor from_preset bug by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1549
• Fix a strange issue with preprocessing layer output types by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1550
• Fix lowercase bug in wordpiece tokenizer by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1543
• Small docs updates by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1553
• Add a few new preset for gemma by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1556
• Remove the dev prefix for 0.9.0 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1557

New Contributors

• @cpsauer made their first contribution in https://github.com/keras-team/keras-nlp/pull/1472
• @SamanehSaadat made their first contribution in https://github.com/keras-team/keras-nlp/pull/1475
• @TheCrazyT made their first contribution in https://github.com/keras-team/keras-nlp/pull/1484
• @shmishra99 made their first contribution in https://github.com/keras-team/keras-nlp/pull/1490
• @sachinprasadhs made their first contribution in https://github.com/keras-team/keras-nlp/pull/1495
• @mykolaskrynnyk made their first contribution in https://github.com/keras-team/keras-nlp/pull/1502
• @RyanMullins made their first contribution in https://github.com/keras-team/keras-nlp/pull/1521
• @Wauplin made their first contribution in https://github.com/keras-team/keras-nlp/pull/1510
• @asmith26 made their first contribution in https://github.com/keras-team/keras-nlp/pull/1528
• @briango28 made their first contribution in https://github.com/keras-team/keras-nlp/pull/1523

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.8.2...v0.9.0