keras-hub | Tensorflow Ecosystem Directory

Bot releases are hidden (Show)

keras-hub - v0.6.0

Published by mattdangerw over 1 year ago

KerasNLP is adding experimental support for Jax and PyTorch backends on top of the Keras Core library. Read the anouncement, and browse the full library documentation, including how to specify the backend when running your code.

Support for both Jax and PyTorch is still experimental, expect some rough edges and please give us feedback!

Summary

This release should be equivalent to 0.5.2 with the addition of multi-backend support.
The following API symbols are currently restricted to the tensorflow backend:
- keras_nlp.models.DebertaV3*
- keras_nlp.models.FNet*
- keras_nlp.metrics
- keras_nlp.samplers.BeamSampler
- keras_nlp.samplers.ContrastiveSampler
Note that there are two ways you can run on top of Tensorflow.
- If you run your scripts/colab without any changes, KerasNLP will use tf.keras for all layer and modeling implementations. This should be a no-op from previous releases of the library.
- If you run your scripts/colab with KERAS_BACKEND={jax, torch, tensorflow}, you will be trying the new Keras Core library, using the specified backend. This is a great way to test out the future of the library!
- Full details on runtime specification is available along with the Keras Core documentation.

What's Changed

small updates to the release doc by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1031
Sampler docstring edit by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1033
Fix program crash for id_to_token() method in SentencePieceTokenizer by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1040
Update our release process to preview docs before release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1043
Add Whisper Tokenizer and Audio Feature Extractor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/847
Also strip padding token for opt by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1028
Add regex dep by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1044
Add BartSeq2SeqLM and conditional text generation with BART by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/974
Support list/tuple inputs for special tokens in StartEndPacker layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1045
Support list/tuple inputs for special tokens in MultiSegmentPacker layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1046
Fix a misleading part of our cached MHA docs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1048
Always pass weight name by kwarg by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1053
Always pass metrics in a list or dict by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1054
Move Defaults to to end of arg docstring and standardise values by @SamuelMarks in https://github.com/keras-team/keras-nlp/pull/1057
Fix beam search for BART by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1058
Replace tf.dtype with "dtype" by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1059
Test shapes directly by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1064
Clean up metrics tests by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1063
Remove metrics merge tests by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1065
Fix whisper feature inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1069
Always specify shape when creating variables by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1067
Remove ragged support from position embeddings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1068
Clean up dtype handling for preprocessing layers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1066
Add BART finetuned on CNN+DM for summarisation by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1060
Fix saving bug by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1073
Fix t5 forward pass by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1082
Feat/make transformer decoder callable without causal mask by @ferraric in https://github.com/keras-team/keras-nlp/pull/1083
Adding GPTNeoXBackbone by @shivance in https://github.com/keras-team/keras-nlp/pull/1056
Add a common test case by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1095
Update register_keras_serializable to use saving module by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1094
Don't test tf format by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1104
Add GPTNeoXPreprocessor by @shivance in https://github.com/keras-team/keras-nlp/pull/1093
Split layers into layers/modeling & layers/preprocessing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1102
Fix merge conflict from #1102 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1105
Add a common base class for generative models by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1096
Add GPTNeoXCausalLMPreprocessor by @shivance in https://github.com/keras-team/keras-nlp/pull/1106
Add Whisper Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1089
Refactor RotaryEmbedding and GPTNeoXAttention by @shivance in https://github.com/keras-team/keras-nlp/pull/1101
Remove all the secret keys for ci by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1126
Fix publish to pypi action by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1127
Unexport models that are not in the 0.6 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1125
Bump the version to 0.6.0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1128

New Contributors

@SamuelMarks made their first contribution in https://github.com/keras-team/keras-nlp/pull/1057
@ferraric made their first contribution in https://github.com/keras-team/keras-nlp/pull/1083

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.5.2...v0.6.0

keras-hub - v0.6.0.dev0

Published by mattdangerw over 1 year ago

🚧 Dev release. 🚧

Support for both Jax and PyTorch is still experimental, expect some rough edges and please give us feedback!

Summary

This release should be equivalent to 0.5.2 with the addition of multi-backend support.
The following API symbols are currently restricted to the tensorflow backend:
- keras_nlp.models.DebertaV3*
- keras_nlp.models.FNet*
- keras_nlp.metrics
- keras_nlp.samplers.BeamSampler
- keras_nlp.samplers.ContrastiveSampler
Note that there are two ways you can run on top of Tensorflow.
- If you run your scripts/colab without any changes, KerasNLP will use tf.keras for all layer and modeling implementations. This should be a no-op from previous releases of the library.
- If you run your scripts/colab with KERAS_BACKEND=tensorflow, you will be trying the new Keras Core library, using the tensorflow backend. This is a great way to test out the future of the library.
- Full details on runtime specification is available along with the Keras Core documentation.

What's Changed

Update python version in readme to 3.8 by @haifeng-jin in https://github.com/keras-team/keras-nlp/pull/618
Modify our pip install line so we upgrade tf by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/616
Use Adam optimizer for quick start by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/620
Clean up class name and self in calls to super() by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Update word_piece_tokenizer.py by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/617
Add DeBERTaV3 Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/633
Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/627
Create Backbone base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
Add TPU testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/591
Add Base Preprocessor Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/638
Add keras_nlp.samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/563
Add ALBERT Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/622
Add a small script to count parameters in our presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/610
Clean up examples/ directory by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/637
Fix Small BERT Typo by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/651
Rename examples/bert -> examples/bert_pretraining by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/647
Add FNet Preprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/646
Add FNet Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/643
Small DeBERTa Docstring Fixes by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/666
Add Fenced Docstring Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/640
Corrected the epsilon value by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/665
Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/654
Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
Add ALBERT Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/655
Add Base Task Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/671
Implement TopP, TopK and Beam samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/652
Add FNet Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/659
Bump the year to 2023 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/679
Add BART Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/661
Handle trainable and name in the backbone base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/680
Ignore Task Docstring for Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/683
Light-weight benchmarking script by @NusretOzates in https://github.com/keras-team/keras-nlp/pull/664
Conditionally import tf_text everywhere by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/684
Expose token_embedding as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
Move from_preset to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/670
import rouge_scorer directly from rouge_score package by @sampathweb in https://github.com/keras-team/keras-nlp/pull/691
Fix typo in requirements file juypter -> jupyter by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/693
Temporary fix to get nightly green again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/696
GPT2 Text Generation APIs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/592
Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/692
Speed up pip install keras-nlp; simplify deps by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/697
Add AlbertClassifier by @shivance in https://github.com/keras-team/keras-nlp/pull/668
Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/700
Update to latest black by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/708
RobertaMaskedLM task and preprocessor by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/653
Default compilation for BERT/RoBERTa classifiers by @jbischof in https://github.com/keras-team/keras-nlp/pull/695
Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
Don't install tf stable when building our nightly image by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/711
Add OPT Backbone and Tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/699
Small OPT Doc-string Edits by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/716
Default compilation other classifiers by @Plutone11011 in https://github.com/keras-team/keras-nlp/pull/714
Add BartTokenizer and BART Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/685
Add an add_prefix_space Arg in BytePairTokenizer by @shivance in https://github.com/keras-team/keras-nlp/pull/715
Opt presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/707
fix import of tensorflow_text in tf_utils by @sampathweb in https://github.com/keras-team/keras-nlp/pull/723
Check for masked token in roberta tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/742
Improve test coverage for special tokens in model tokenizers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/743
Fix the sampler truncation strategy by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/713
Add ALBERT Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/736
Add FNet Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/737
Add BART Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/739
Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/731
Improving the layer Description. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/734
Adding ragged support to SinePositionEncoding by @apupneja in https://github.com/keras-team/keras-nlp/pull/751
Fix trailing space by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/755
Adding an AlbertMaskedLM task + Fix Projection layer dimension in MaskedLMHead by @shivance in https://github.com/keras-team/keras-nlp/pull/725
New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/760
Add a note for TPU issues for deberta_v3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/758
Add missing exports to models API by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/763
Autogenerate preset table by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/690
Version bump to 0.5.0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/767
Adding a FNetMaskedLM task model and preprocessor by @apupneja in https://github.com/keras-team/keras-nlp/pull/740
Add a DistilBertMaskedLM task model by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/724
Add cache support to decoding journey by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/745
Handle [MASK] token in DebertaV3Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/759
Update README for 2.4.1 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/757
Fix typo in test docstring by @jbischof in https://github.com/keras-team/keras-nlp/pull/791
Fixed Incorrect Links for FNet and DeBERTaV3 models by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/793
Patch 1 - doc-string spell fix by @atharvapurdue in https://github.com/keras-team/keras-nlp/pull/781
Don't rely on core keras initializer config details by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/802
Simplify the cache decoding graph by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/780
Fix Fenced Doc-String #782 by @atharvapurdue in https://github.com/keras-team/keras-nlp/pull/785
Solve #721 Deberta masklm model by @Plutone11011 in https://github.com/keras-team/keras-nlp/pull/732
Add from_config to sampler by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/803
BertMaskedLM Task Model and Preprocessor by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/774
Stop generation once end_token_id is seen by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/769
Added model card links for all pretrained models. by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/795
Initial PR demonstrating public API export logic. by @fchollet in https://github.com/keras-team/keras-nlp/pull/747
Add preset for finetuning GPT2 on CNN news by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/807
Add API exports for metrics documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/816
Add API exports for samplers documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/815
Add API exports for models documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/814
Add API exports for tokenizers documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/817
Add API exports for layers documented on keras.io by @fchollet in https://github.com/keras-team/keras-nlp/pull/811
Add keras_nlp.utils public API exports. by @fchollet in https://github.com/keras-team/keras-nlp/pull/819
retrained bert_tiny_uncased_en_sst2_training.ipynb by @susnato in https://github.com/keras-team/keras-nlp/pull/771
Temporary solution to avoid recompilation by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/808
Call super.config() in BartBackbone's get_config() by @shivance in https://github.com/keras-team/keras-nlp/pull/818
Update typo in README.md by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/821
Add Whisper Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/801
Added note for tensorflow-text in the CONTRIBUTING guide by @jaygala223 in https://github.com/keras-team/keras-nlp/pull/805
Roadmap update by @jaygala223 in https://github.com/keras-team/keras-nlp/pull/800
Remove API export decorator from base classes by @shivance in https://github.com/keras-team/keras-nlp/pull/824
Move integration tests out of repo sources. by @fchollet in https://github.com/keras-team/keras-nlp/pull/826
Function merge_padding_and_attention_mask does not return an output with the desired shape when both padding and attention masks are given by @abodinier in https://github.com/keras-team/keras-nlp/pull/790
Adding XXBackboneTPUTests by @shivance in https://github.com/keras-team/keras-nlp/pull/839
Add a t5 tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/852
Add compilation defaults for the BertMaskedLM task model by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/836
added init file for t5 by @Akorex in https://github.com/keras-team/keras-nlp/pull/853
Modified Docstring for GPT2CasualLM by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/855
Rework bert docstrings for progressive disclosure of complexity by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/843
Fix "causal" spelling in export decorator by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/861
Default compilation for Albert, Distilbert, Roberta MaskedLM by @shivance in https://github.com/keras-team/keras-nlp/pull/833
Speed up default BERT testing roughly 3x by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/859
Add compilation defaults for the Fnet MaskedLM task model by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/834
Default compilation for Debertav3MaskedLM model by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/835
Remove from_preset from fnet tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/865
Add T5 backbone by @fchollet in https://github.com/keras-team/keras-nlp/pull/828
Speeding the tests for opt by @susnato in https://github.com/keras-team/keras-nlp/pull/886
Move generate compilation to the task model by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/804
Speeding the tests for xlm_roberta by @susnato in https://github.com/keras-team/keras-nlp/pull/885
Rework DistilBERT docstrings for progressive disclosure of complexity. by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/881
Speeding the tests for T5 by @susnato in https://github.com/keras-team/keras-nlp/pull/888
Rework OPT docstrings for progressive disclosure of complexity. by @Warlord-K in https://github.com/keras-team/keras-nlp/pull/893
Get our fenced docstring tests working again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/895
Speed up default RoBERTa testing roughly 3x by @shivance in https://github.com/keras-team/keras-nlp/pull/897
Speeding the tests for whisper by @susnato in https://github.com/keras-team/keras-nlp/pull/887
Update BytePairTokenizerCache to have similar dtypes for x and y in self.factors. by @Sruinard in https://github.com/keras-team/keras-nlp/pull/871
Init _backbone, _tokenizer and _preprocessor in Task by @jbischof in https://github.com/keras-team/keras-nlp/pull/899
Rework Whisper docstrings for progressive disclosure of complexity by @susnato in https://github.com/keras-team/keras-nlp/pull/903
Speed up default DeBERTa_v3 testing roughly 3x by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/905
Rework docstring of XLMRoberta by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/882
Stripping the MASK token by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/876
Possible fix for task.summary() by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/901
Speed up default FNet testing speedups. by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/894
Added TPU test for DebertaV3Backbone by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/924
Fix failing TPU tests by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/931
Add model contribution guide by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/820
Resolved roberta_checkpoint by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/874
GLUE evaluation automation script by @susnato in https://github.com/keras-team/keras-nlp/pull/848
Ensure shape in sample so that the shape is correct after TFLite conversion by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/902
Returning all Beams and Probs and adding a Testing Unit by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/908
Roberta docstring reworking by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/910
Speeding the tests for Albert by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/873
Mlm mask generator docstring adding example by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/916
Don't save traces for saved model by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/945
Bump stable tf version to 2.12 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/944
Speeding the tests for DistilBert by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/872
Allow BPE to treat special tokens as one token by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/939
Edit examples in samplers by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/957
Add RandomSampler to Samplers by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/952
Add BartPreprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/856
Remove the old sampler utilities by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/948
Use direct imports everywhere in library by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/961
Update docstrings for relocated sampler arg by @jbischof in https://github.com/keras-team/keras-nlp/pull/964
Fix gpt2, t5 and fnet under mixed precision by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/958
Small fixes for special_tokens arg in BPE by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/969
Add contrastive sampler by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/896
Mark num_classes as required in Classifier classes by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/971
Rework model docstrings for progressive disclosure of complexity for f_net by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/879
Handle OOV token in XLMRoBERTaTokenizer's token_to_id function by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/968
Clean up the docker and lint setup by @haifeng-jin in https://github.com/keras-team/keras-nlp/pull/981
Update generate() to work like fit() and predict() by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/932
Speed top-p sampler up by only sampling from top-k tokens by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/980
Expose the generate_step compilable function by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/982
Fix decoder inputs in BART preprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/984
Convert string tensors to python strings in generate() by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/983
Adding a temperature argument to the base sampler class and related tests by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/951
Track the task preprocessor layer as part of model by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/985
Add an XLMRobertaMaskedLM task model by @shivance in https://github.com/keras-team/keras-nlp/pull/950
Add an activation argument to all classifiers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/991
Remove activation from README quickstart by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/992
Rework albert docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/993
Rework bart docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/994
Rework deberta docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/995
Misc fixes to docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/996
Added temperature argument to the Contrastive Sampler by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/997
Add OPTCausalLM and preprocessors by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/990
Version bump to 0.5.0.dev0 by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1002
Add a flag to restrict which docstring tests run by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/999
fix docstring for 0.5 release by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1005
Serialize activation fn properly by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1007
Try adding an error if activation and loss are mismatched by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1008
Fix docstring for 0.5 release by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1009
Switch to using pip_build for release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1011
Make version number SSoT. by @fchollet in https://github.com/keras-team/keras-nlp/pull/827
Add DTensor layout map class method for OPT by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1000
Add DTensor layout map class method for GPT-2 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1014
Standalone functions for generate pre/post processing for GPT-2 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/998
install namex in the publish workflow by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1020
Update publish-to-pypi.yml by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1021
Standalone functions for generate pre/post processing for OPT by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1015
Fix typos in export by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1024
Fix unclosed fenced docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1025
Fix a bug with computing the output mask after generate by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1029
small updates to the release doc by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1031
Sampler docstring edit by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1033
Fix program crash for id_to_token() method in SentencePieceTokenizer by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/1040
Update our release process to preview docs before release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1043
Add Whisper Tokenizer and Audio Feature Extractor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/847
Also strip padding token for opt by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1028
Add regex dep by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1044
Add BartSeq2SeqLM and conditional text generation with BART by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/974
Support list/tuple inputs for special tokens in StartEndPacker layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1045
Support list/tuple inputs for special tokens in MultiSegmentPacker layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1046
Fix a misleading part of our cached MHA docs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1048
Always pass weight name by kwarg by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1053
Always pass metrics in a list or dict by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1054
Move Defaults to to end of arg docstring and standardise values by @SamuelMarks in https://github.com/keras-team/keras-nlp/pull/1057
Fix beam search for BART by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1058
Replace tf.dtype with "dtype" by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1059
Test shapes directly by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1064
Clean up metrics tests by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1063
Remove metrics merge tests by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1065
Fix whisper feature inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1069
Always specify shape when creating variables by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1067
Remove ragged support from position embeddings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1068
Clean up dtype handling for preprocessing layers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1066
Add BART finetuned on CNN+DM for summarisation by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1060
Fix saving bug by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1073
Fix t5 forward pass by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1082
Feat/make transformer decoder callable without causal mask by @ferraric in https://github.com/keras-team/keras-nlp/pull/1083
Adding GPTNeoXBackbone by @shivance in https://github.com/keras-team/keras-nlp/pull/1056
Add a common test case by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1095
Update register_keras_serializable to use saving module by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1094
Don't test tf format by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1104
Add GPTNeoXPreprocessor by @shivance in https://github.com/keras-team/keras-nlp/pull/1093
Split layers into layers/modeling & layers/preprocessing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1102
Fix merge conflict from #1102 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1105
Add a common base class for generative models by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1096
Add GPTNeoXCausalLMPreprocessor by @shivance in https://github.com/keras-team/keras-nlp/pull/1106
Add Whisper Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/1089
Refactor RotaryEmbedding and GPTNeoXAttention by @shivance in https://github.com/keras-team/keras-nlp/pull/1101
Remove all the secret keys for ci by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1126
Fix publish to pypi action by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1127
Unexport models that are not in the 0.6 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1125

New Contributors

@haifeng-jin made their first contribution in https://github.com/keras-team/keras-nlp/pull/618
@mbrukman made their first contribution in https://github.com/keras-team/keras-nlp/pull/628
@soma2000-lang made their first contribution in https://github.com/keras-team/keras-nlp/pull/665
@NusretOzates made their first contribution in https://github.com/keras-team/keras-nlp/pull/664
@shivance made their first contribution in https://github.com/keras-team/keras-nlp/pull/673
@Plutone11011 made their first contribution in https://github.com/keras-team/keras-nlp/pull/714
@TheAthleticCoder made their first contribution in https://github.com/keras-team/keras-nlp/pull/731
@Neeshamraghav012 made their first contribution in https://github.com/keras-team/keras-nlp/pull/734
@apupneja made their first contribution in https://github.com/keras-team/keras-nlp/pull/751
@Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/690
@atharvapurdue made their first contribution in https://github.com/keras-team/keras-nlp/pull/781
@fchollet made their first contribution in https://github.com/keras-team/keras-nlp/pull/747
@susnato made their first contribution in https://github.com/keras-team/keras-nlp/pull/771
@jaygala223 made their first contribution in https://github.com/keras-team/keras-nlp/pull/805
@abodinier made their first contribution in https://github.com/keras-team/keras-nlp/pull/790
@Akorex made their first contribution in https://github.com/keras-team/keras-nlp/pull/853
@Warlord-K made their first contribution in https://github.com/keras-team/keras-nlp/pull/893
@Sruinard made their first contribution in https://github.com/keras-team/keras-nlp/pull/871
@abuelnasr0 made their first contribution in https://github.com/keras-team/keras-nlp/pull/882
@SamuelMarks made their first contribution in https://github.com/keras-team/keras-nlp/pull/1057
@ferraric made their first contribution in https://github.com/keras-team/keras-nlp/pull/1083

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.6.0.dev0

keras-hub - v0.5.2

Published by chenmoneygithub over 1 year ago

What's Changed

Fix unclosed fenced docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1025
Fix a bug with computing the output mask after generate by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1029

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.5.1...v0.5.2

keras-hub - v0.5.1

Published by chenmoneygithub over 1 year ago

What's Changed

Fix typos in export by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1024

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.5.0...v0.5.1

keras-hub - v0.5.0

Published by chenmoneygithub over 1 year ago

In this 0.5 release, we are bringing generative AI to KerasNLP!

Summary

Added text generation task model keras_nlp.models.GPT2CausalLM and keras_nlp.models.OPTCausalLM along with corresponding preprocessors. Both task models exposed a public generate() method for text generation.
Refactored text generation utils into sampler APIs in keras_nlp.samplers for better UX and scalability.
Added MaskedLM task models keras_nlp.models.XXXMaskedLM, e.g., keras_nlp.models.BertMaskedLM.

What's Changed

Update python version in readme to 3.8 by @haifeng-jin in https://github.com/keras-team/keras-nlp/pull/618
Modify our pip install line so we upgrade tf by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/616
Use Adam optimizer for quick start by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/620
Clean up class name and self in calls to super() by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Update word_piece_tokenizer.py by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/617
Add DeBERTaV3 Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/633
Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/627
Create Backbone base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
Add TPU testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/591
Add Base Preprocessor Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/638
Add keras_nlp.samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/563
Add ALBERT Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/622
Add a small script to count parameters in our presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/610
Clean up examples/ directory by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/637
Fix Small BERT Typo by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/651
Rename examples/bert -> examples/bert_pretraining by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/647
Add FNet Preprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/646
Add FNet Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/643
Small DeBERTa Docstring Fixes by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/666
Add Fenced Docstring Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/640
Corrected the epsilon value by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/665
Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/654
Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
Add ALBERT Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/655
Add Base Task Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/671
Implement TopP, TopK and Beam samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/652
Add FNet Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/659
Bump the year to 2023 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/679
Add BART Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/661
Handle trainable and name in the backbone base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/680
Ignore Task Docstring for Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/683
Light-weight benchmarking script by @NusretOzates in https://github.com/keras-team/keras-nlp/pull/664
Conditionally import tf_text everywhere by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/684
Expose token_embedding as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
Move from_preset to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/670
import rouge_scorer directly from rouge_score package by @sampathweb in https://github.com/keras-team/keras-nlp/pull/691
Fix typo in requirements file juypter -> jupyter by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/693
Temporary fix to get nightly green again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/696
GPT2 Text Generation APIs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/592
Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/692
Speed up pip install keras-nlp; simplify deps by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/697
Add AlbertClassifier by @shivance in https://github.com/keras-team/keras-nlp/pull/668
Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/700
Update to latest black by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/708
RobertaMaskedLM task and preprocessor by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/653
Default compilation for BERT/RoBERTa classifiers by @jbischof in https://github.com/keras-team/keras-nlp/pull/695
Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
Don't install tf stable when building our nightly image by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/711
Add OPT Backbone and Tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/699
Small OPT Doc-string Edits by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/716
Default compilation other classifiers by @Plutone11011 in https://github.com/keras-team/keras-nlp/pull/714
Add BartTokenizer and BART Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/685
Add an add_prefix_space Arg in BytePairTokenizer by @shivance in https://github.com/keras-team/keras-nlp/pull/715
Opt presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/707
fix import of tensorflow_text in tf_utils by @sampathweb in https://github.com/keras-team/keras-nlp/pull/723
Check for masked token in roberta tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/742
Improve test coverage for special tokens in model tokenizers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/743
Fix the sampler truncation strategy by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/713
Add ALBERT Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/736
Add FNet Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/737
Add BART Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/739
Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/731
Improving the layer Description. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/734
Adding ragged support to SinePositionEncoding by @apupneja in https://github.com/keras-team/keras-nlp/pull/751
Fix trailing space by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/755
Adding an AlbertMaskedLM task + Fix Projection layer dimension in MaskedLMHead by @shivance in https://github.com/keras-team/keras-nlp/pull/725
New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/760
Add a note for TPU issues for deberta_v3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/758
Add missing exports to models API by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/763
Autogenerate preset table by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/690
Version bump to 0.5.0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/767
Adding a FNetMaskedLM task model and preprocessor by @apupneja in https://github.com/keras-team/keras-nlp/pull/740
Add a DistilBertMaskedLM task model by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/724
Add cache support to decoding journey by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/745
Handle [MASK] token in DebertaV3Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/759
Update README for 2.4.1 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/757
Fix typo in test docstring by @jbischof in https://github.com/keras-team/keras-nlp/pull/791
Fixed Incorrect Links for FNet and DeBERTaV3 models by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/793
Patch 1 - doc-string spell fix by @atharvapurdue in https://github.com/keras-team/keras-nlp/pull/781
Don't rely on core keras initializer config details by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/802
Simplify the cache decoding graph by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/780
Fix Fenced Doc-String #782 by @atharvapurdue in https://github.com/keras-team/keras-nlp/pull/785
Solve #721 Deberta masklm model by @Plutone11011 in https://github.com/keras-team/keras-nlp/pull/732
Add from_config to sampler by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/803
BertMaskedLM Task Model and Preprocessor by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/774
Stop generation once end_token_id is seen by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/769
Added model card links for all pretrained models. by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/795
Initial PR demonstrating public API export logic. by @fchollet in https://github.com/keras-team/keras-nlp/pull/747
Add preset for finetuning GPT2 on CNN news by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/807
Add API exports for metrics documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/816
Add API exports for samplers documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/815
Add API exports for models documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/814
Add API exports for tokenizers documented on keras.io by @shivance in https://github.com/keras-team/keras-nlp/pull/817
Add API exports for layers documented on keras.io by @fchollet in https://github.com/keras-team/keras-nlp/pull/811
Add keras_nlp.utils public API exports. by @fchollet in https://github.com/keras-team/keras-nlp/pull/819
retrained bert_tiny_uncased_en_sst2_training.ipynb by @susnato in https://github.com/keras-team/keras-nlp/pull/771
Temporary solution to avoid recompilation by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/808
Call super.config() in BartBackbone's get_config() by @shivance in https://github.com/keras-team/keras-nlp/pull/818
Update typo in README.md by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/821
Add Whisper Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/801
Added note for tensorflow-text in the CONTRIBUTING guide by @jaygala223 in https://github.com/keras-team/keras-nlp/pull/805
Roadmap update by @jaygala223 in https://github.com/keras-team/keras-nlp/pull/800
Remove API export decorator from base classes by @shivance in https://github.com/keras-team/keras-nlp/pull/824
Move integration tests out of repo sources. by @fchollet in https://github.com/keras-team/keras-nlp/pull/826
Function merge_padding_and_attention_mask does not return an output with the desired shape when both padding and attention masks are given by @abodinier in https://github.com/keras-team/keras-nlp/pull/790
Adding XXBackboneTPUTests by @shivance in https://github.com/keras-team/keras-nlp/pull/839
Add a t5 tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/852
Add compilation defaults for the BertMaskedLM task model by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/836
added init file for t5 by @Akorex in https://github.com/keras-team/keras-nlp/pull/853
Modified Docstring for GPT2CasualLM by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/855
Rework bert docstrings for progressive disclosure of complexity by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/843
Fix "causal" spelling in export decorator by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/861
Default compilation for Albert, Distilbert, Roberta MaskedLM by @shivance in https://github.com/keras-team/keras-nlp/pull/833
Speed up default BERT testing roughly 3x by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/859
Add compilation defaults for the Fnet MaskedLM task model by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/834
Default compilation for Debertav3MaskedLM model by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/835
Remove from_preset from fnet tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/865
Add T5 backbone by @fchollet in https://github.com/keras-team/keras-nlp/pull/828
Speeding the tests for opt by @susnato in https://github.com/keras-team/keras-nlp/pull/886
Move generate compilation to the task model by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/804
Speeding the tests for xlm_roberta by @susnato in https://github.com/keras-team/keras-nlp/pull/885
Rework DistilBERT docstrings for progressive disclosure of complexity. by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/881
Speeding the tests for T5 by @susnato in https://github.com/keras-team/keras-nlp/pull/888
Rework OPT docstrings for progressive disclosure of complexity. by @Warlord-K in https://github.com/keras-team/keras-nlp/pull/893
Get our fenced docstring tests working again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/895
Speed up default RoBERTa testing roughly 3x by @shivance in https://github.com/keras-team/keras-nlp/pull/897
Speeding the tests for whisper by @susnato in https://github.com/keras-team/keras-nlp/pull/887
Update BytePairTokenizerCache to have similar dtypes for x and y in self.factors. by @Sruinard in https://github.com/keras-team/keras-nlp/pull/871
Init _backbone, _tokenizer and _preprocessor in Task by @jbischof in https://github.com/keras-team/keras-nlp/pull/899
Rework Whisper docstrings for progressive disclosure of complexity by @susnato in https://github.com/keras-team/keras-nlp/pull/903
Speed up default DeBERTa_v3 testing roughly 3x by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/905
Rework docstring of XLMRoberta by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/882
Stripping the MASK token by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/876
Possible fix for task.summary() by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/901
Speed up default FNet testing speedups. by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/894
Added TPU test for DebertaV3Backbone by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/924
Fix failing TPU tests by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/931
Add model contribution guide by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/820
Resolved roberta_checkpoint by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/874
GLUE evaluation automation script by @susnato in https://github.com/keras-team/keras-nlp/pull/848
Ensure shape in sample so that the shape is correct after TFLite conversion by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/902
Returning all Beams and Probs and adding a Testing Unit by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/908
Roberta docstring reworking by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/910
Speeding the tests for Albert by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/873
Mlm mask generator docstring adding example by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/916
Don't save traces for saved model by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/945
Bump stable tf version to 2.12 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/944
Speeding the tests for DistilBert by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/872
Allow BPE to treat special tokens as one token by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/939
Edit examples in samplers by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/957
Add RandomSampler to Samplers by @abuelnasr0 in https://github.com/keras-team/keras-nlp/pull/952
Add BartPreprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/856
Remove the old sampler utilities by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/948
Use direct imports everywhere in library by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/961
Update docstrings for relocated sampler arg by @jbischof in https://github.com/keras-team/keras-nlp/pull/964
Fix gpt2, t5 and fnet under mixed precision by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/958
Small fixes for special_tokens arg in BPE by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/969
Add contrastive sampler by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/896
Mark num_classes as required in Classifier classes by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/971
Rework model docstrings for progressive disclosure of complexity for f_net by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/879
Handle OOV token in XLMRoBERTaTokenizer's token_to_id function by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/968
Clean up the docker and lint setup by @haifeng-jin in https://github.com/keras-team/keras-nlp/pull/981
Update generate() to work like fit() and predict() by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/932
Speed top-p sampler up by only sampling from top-k tokens by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/980
Expose the generate_step compilable function by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/982
Fix decoder inputs in BART preprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/984
Convert string tensors to python strings in generate() by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/983
Adding a temperature argument to the base sampler class and related tests by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/951
Track the task preprocessor layer as part of model by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/985
Add an XLMRobertaMaskedLM task model by @shivance in https://github.com/keras-team/keras-nlp/pull/950
Add an activation argument to all classifiers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/991
Remove activation from README quickstart by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/992
Rework albert docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/993
Rework bart docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/994
Rework deberta docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/995
Misc fixes to docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/996
Added temperature argument to the Contrastive Sampler by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/997
Add OPTCausalLM and preprocessors by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/990
Version bump to 0.5.0.dev0 by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1002
Add a flag to restrict which docstring tests run by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/999
fix docstring for 0.5 release by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1005
Serialize activation fn properly by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1007
Try adding an error if activation and loss are mismatched by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1008
Fix docstring for 0.5 release by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1009
Switch to using pip_build for release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1011
Make version number SSoT. by @fchollet in https://github.com/keras-team/keras-nlp/pull/827
Add DTensor layout map class method for OPT by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1000
Add DTensor layout map class method for GPT-2 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1014
Standalone functions for generate pre/post processing for GPT-2 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/998
install namex in the publish workflow by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1020
Update publish-to-pypi.yml by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/1021
Standalone functions for generate pre/post processing for OPT by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/1015

New Contributors

@haifeng-jin made their first contribution in https://github.com/keras-team/keras-nlp/pull/618
@mbrukman made their first contribution in https://github.com/keras-team/keras-nlp/pull/628
@soma2000-lang made their first contribution in https://github.com/keras-team/keras-nlp/pull/665
@NusretOzates made their first contribution in https://github.com/keras-team/keras-nlp/pull/664
@shivance made their first contribution in https://github.com/keras-team/keras-nlp/pull/673
@Plutone11011 made their first contribution in https://github.com/keras-team/keras-nlp/pull/714
@TheAthleticCoder made their first contribution in https://github.com/keras-team/keras-nlp/pull/731
@Neeshamraghav012 made their first contribution in https://github.com/keras-team/keras-nlp/pull/734
@apupneja made their first contribution in https://github.com/keras-team/keras-nlp/pull/751
@Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/690
@atharvapurdue made their first contribution in https://github.com/keras-team/keras-nlp/pull/781
@fchollet made their first contribution in https://github.com/keras-team/keras-nlp/pull/747
@susnato made their first contribution in https://github.com/keras-team/keras-nlp/pull/771
@jaygala223 made their first contribution in https://github.com/keras-team/keras-nlp/pull/805
@abodinier made their first contribution in https://github.com/keras-team/keras-nlp/pull/790
@Akorex made their first contribution in https://github.com/keras-team/keras-nlp/pull/853
@Warlord-K made their first contribution in https://github.com/keras-team/keras-nlp/pull/893
@Sruinard made their first contribution in https://github.com/keras-team/keras-nlp/pull/871
@abuelnasr0 made their first contribution in https://github.com/keras-team/keras-nlp/pull/882

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.5.0

keras-hub - v0.4.1

Published by mattdangerw over 1 year ago

The 0.4.1 release is a minor release with new model architectures and compilation defaults for task models. If you encounter any problems or have questions, please open an issue!

Summary

Added compilation defaults for all task models (e.g. keras_nlp.models.BertClassifier). No existing functionality is changed, but users of task models can now skip calling .compile() and use default learning rates and optimization strategies provided by the library.
Added keras_nlp.models.AlbertBackbone, keras_nlp.models.AlbertClassifier, preprocessor, and tokenizer layers for pre-trained ALBERT models.
Added keras_nlp.models.FNetBackbone, keras_nlp.models.FNetClassifier, preprocessor, and tokenizer layers for pre-trained FNet models.
Added keras_nlp.models.DebertaV3Backbone, keras_nlp.models.DebertaV3Classifier, preprocessor, and tokenizer layers for pre-trained DeBERTaV3 models.

What's Changed

Update python version in readme to 3.8 by @haifeng-jin in https://github.com/keras-team/keras-nlp/pull/618
Modify our pip install line so we upgrade tf by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/616
Use Adam optimizer for quick start by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/620
Clean up class name and self in calls to super() by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Update word_piece_tokenizer.py by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/617
Add DeBERTaV3 Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/633
Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/627
Create Backbone base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
Add TPU testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/591
Add Base Preprocessor Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/638
Add keras_nlp.samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/563
Add ALBERT Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/622
Add a small script to count parameters in our presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/610
Clean up examples/ directory by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/637
Fix Small BERT Typo by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/651
Rename examples/bert -> examples/bert_pretraining by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/647
Add FNet Preprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/646
Add FNet Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/643
Small DeBERTa Docstring Fixes by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/666
Add Fenced Docstring Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/640
Corrected the epsilon value by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/665
Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/654
Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
Add ALBERT Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/655
Add Base Task Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/671
Implement TopP, TopK and Beam samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/652
Add FNet Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/659
Bump the year to 2023 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/679
Add BART Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/661
Handle trainable and name in the backbone base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/680
Ignore Task Docstring for Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/683
Light-weight benchmarking script by @NusretOzates in https://github.com/keras-team/keras-nlp/pull/664
Conditionally import tf_text everywhere by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/684
Expose token_embedding as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
Move from_preset to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/670
import rouge_scorer directly from rouge_score package by @sampathweb in https://github.com/keras-team/keras-nlp/pull/691
Fix typo in requirements file juypter -> jupyter by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/693
Temporary fix to get nightly green again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/696
GPT2 Text Generation APIs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/592
Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/692
Speed up pip install keras-nlp; simplify deps by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/697
Add AlbertClassifier by @shivance in https://github.com/keras-team/keras-nlp/pull/668
Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/700
Update to latest black by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/708
RobertaMaskedLM task and preprocessor by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/653
Default compilation for BERT/RoBERTa classifiers by @jbischof in https://github.com/keras-team/keras-nlp/pull/695
Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
Don't install tf stable when building our nightly image by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/711
Add OPT Backbone and Tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/699
Small OPT Doc-string Edits by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/716
Default compilation other classifiers by @Plutone11011 in https://github.com/keras-team/keras-nlp/pull/714
Add BartTokenizer and BART Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/685
Add an add_prefix_space Arg in BytePairTokenizer by @shivance in https://github.com/keras-team/keras-nlp/pull/715
Opt presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/707
fix import of tensorflow_text in tf_utils by @sampathweb in https://github.com/keras-team/keras-nlp/pull/723
Check for masked token in roberta tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/742
Improve test coverage for special tokens in model tokenizers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/743
Fix the sampler truncation strategy by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/713
Add ALBERT Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/736
Add FNet Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/737
Add BART Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/739
Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/731
Improving the layer Description. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/734
Adding ragged support to SinePositionEncoding by @apupneja in https://github.com/keras-team/keras-nlp/pull/751
Fix trailing space by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/755
Adding an AlbertMaskedLM task model and preprocessor by @shivance in https://github.com/keras-team/keras-nlp/pull/725
New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/760
Add a note for TPU issues for deberta_v3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/758
Add missing exports to models API by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/763
Autogenerate preset table by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/690
Remove work in progress API for 0.4.1 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/765
Version bump to 0.4.1.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/766
Version bump to 0.4.1 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/768

New Contributors

@haifeng-jin made their first contribution in https://github.com/keras-team/keras-nlp/pull/618
@mbrukman made their first contribution in https://github.com/keras-team/keras-nlp/pull/628
@soma2000-lang made their first contribution in https://github.com/keras-team/keras-nlp/pull/665
@NusretOzates made their first contribution in https://github.com/keras-team/keras-nlp/pull/664
@shivance made their first contribution in https://github.com/keras-team/keras-nlp/pull/673
@Plutone11011 made their first contribution in https://github.com/keras-team/keras-nlp/pull/714
@TheAthleticCoder made their first contribution in https://github.com/keras-team/keras-nlp/pull/731
@Neeshamraghav012 made their first contribution in https://github.com/keras-team/keras-nlp/pull/734
@Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/690

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.4.1

keras-hub - r0.4.1.dev0

Published by mattdangerw over 1 year ago

Summary

Dev release to test out the upcoming 0.4.1.

What's Changed

Update python version in readme to 3.8 by @haifeng-jin in https://github.com/keras-team/keras-nlp/pull/618
Modify our pip install line so we upgrade tf by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/616
Use Adam optimizer for quick start by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/620
Clean up class name and self in calls to super() by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Update word_piece_tokenizer.py by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/617
Add DeBERTaV3 Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/633
Add AlbertTokenizer and AlbertPreprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/627
Create Backbone base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
Add TPU testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/591
Add Base Preprocessor Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/638
Add keras_nlp.samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/563
Add ALBERT Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/622
Add a small script to count parameters in our presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/610
Clean up examples/ directory by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/637
Fix Small BERT Typo by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/651
Rename examples/bert -> examples/bert_pretraining by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/647
Add FNet Preprocessor by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/646
Add FNet Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/643
Small DeBERTa Docstring Fixes by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/666
Add Fenced Docstring Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/640
Corrected the epsilon value by @soma2000-lang in https://github.com/keras-team/keras-nlp/pull/665
Consolidate docstring formatting weirdness in Backbone and Preprocessor base classes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/654
Fix value_dim in TransformerDecoder's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
Add ALBERT Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/655
Add Base Task Class by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/671
Implement TopP, TopK and Beam samplers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/652
Add FNet Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/659
Bump the year to 2023 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/679
Add BART Backbone by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/661
Handle trainable and name in the backbone base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/680
Ignore Task Docstring for Testing by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/683
Light-weight benchmarking script by @NusretOzates in https://github.com/keras-team/keras-nlp/pull/664
Conditionally import tf_text everywhere by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/684
Expose token_embedding as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
Move from_preset to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
add f_net_classifier and f_net_classifier_test by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/670
import rouge_scorer directly from rouge_score package by @sampathweb in https://github.com/keras-team/keras-nlp/pull/691
Fix typo in requirements file juypter -> jupyter by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/693
Temporary fix to get nightly green again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/696
GPT2 Text Generation APIs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/592
Run keras saving tests on nightly and fix RobertaClassifier test by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/692
Speed up pip install keras-nlp; simplify deps by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/697
Add AlbertClassifier by @shivance in https://github.com/keras-team/keras-nlp/pull/668
Make tokenizer, backbone, preprocessor properties settable on base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/700
Update to latest black by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/708
RobertaMaskedLM task and preprocessor by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/653
Default compilation for BERT/RoBERTa classifiers by @jbischof in https://github.com/keras-team/keras-nlp/pull/695
Add start/end token padding to GPT2Preprocessor by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
Don't install tf stable when building our nightly image by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/711
Add OPT Backbone and Tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/699
Small OPT Doc-string Edits by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/716
Default compilation other classifiers by @Plutone11011 in https://github.com/keras-team/keras-nlp/pull/714
Add BartTokenizer and BART Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/685
Add an add_prefix_space Arg in BytePairTokenizer by @shivance in https://github.com/keras-team/keras-nlp/pull/715
Opt presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/707
fix import of tensorflow_text in tf_utils by @sampathweb in https://github.com/keras-team/keras-nlp/pull/723
Check for masked token in roberta tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/742
Improve test coverage for special tokens in model tokenizers by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/743
Fix the sampler truncation strategy by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/713
Add ALBERT Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/736
Add FNet Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/737
Add BART Conversion Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/739
Pass Correct LayerNorm Epsilon value to TransformerEncoder in Backbones by @TheAthleticCoder in https://github.com/keras-team/keras-nlp/pull/731
Improving the layer Description. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/734
Adding ragged support to SinePositionEncoding by @apupneja in https://github.com/keras-team/keras-nlp/pull/751
Fix trailing space by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/755
Adding an AlbertMaskedLM task model and preprocessor by @shivance in https://github.com/keras-team/keras-nlp/pull/725
New docstring example for TokenAndPosition Embedding layer. by @Neeshamraghav012 in https://github.com/keras-team/keras-nlp/pull/760
Add a note for TPU issues for deberta_v3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/758
Add missing exports to models API by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/763
Autogenerate preset table by @Cyber-Machine in https://github.com/keras-team/keras-nlp/pull/690
Version bump to 0.5.0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/767

New Contributors

@haifeng-jin made their first contribution in https://github.com/keras-team/keras-nlp/pull/618
@mbrukman made their first contribution in https://github.com/keras-team/keras-nlp/pull/628
@soma2000-lang made their first contribution in https://github.com/keras-team/keras-nlp/pull/665
@NusretOzates made their first contribution in https://github.com/keras-team/keras-nlp/pull/664
@shivance made their first contribution in https://github.com/keras-team/keras-nlp/pull/673
@Plutone11011 made their first contribution in https://github.com/keras-team/keras-nlp/pull/714
@TheAthleticCoder made their first contribution in https://github.com/keras-team/keras-nlp/pull/731
@Neeshamraghav012 made their first contribution in https://github.com/keras-team/keras-nlp/pull/734
@apupneja made their first contribution in https://github.com/keras-team/keras-nlp/pull/751
@Cyber-Machine made their first contribution in https://github.com/keras-team/keras-nlp/pull/690

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.4.1.dev0

keras-hub - v0.4.0

Published by mattdangerw almost 2 years ago

The 0.4 release adds support for pretrained models to the library via keras_nlp.models. You can read an
introduction to the new API in our Getting Started Guide.

If you encounter any problems or have questions, please open an issue!

Breaking Changes

Renamed keras_nlp.layers.MLMHead -> keras_nlp.layers.MaskedLMHead.
Renamed keras_nlp.layers.MLMMaskGenerator -> keras_nlp.layers.MaskedLMMaskGenerator.
Renamed keras_nlp.layers.UnicodeCharacterTokenizer -> keras_nlp.layers.UnicodeCodepointTokenizer.
Switched the default of lowercase in keras_nlp.tokenizers.WordPieceTokenizer from True to False.
Renamed the token id output of MaskedLMMaskGenerator from "tokens" to "tokens_ids".

Summary

Added the keras_nlp.models API.
- Added support for BERT, DistilBERT, RoBERTa, and XLM-RoBERTa models and pretrained checkpoints.
- See our Getting Started Guide for more details.
Added new metrics.
- keras_nlp.metrics.Bleu and keras_nlp.metrics.EditDistance.
Added new vocabulary training utilities.
- keras_nlp.tokenizers.compute_word_piece_vocabulary and keras_nlp.tokenizers.compute_sentence_piece_proto.
Added new preprocessing layers.
- keras_nlp.layers.RandomSwap and keras_nlp.layers.RandomDeletion.

What's Changed

Add Edit Distance Metric by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/231
Minor fix to simplify and test handling of max_length prompts by @jbischof in https://github.com/keras-team/keras-nlp/pull/258
Remove split regex args for WordPieceTokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/255
Add instructions on installing the latest changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/261
Add warning when k > vocab_size in top_k_search by @jbischof in https://github.com/keras-team/keras-nlp/pull/260
Fix keras library imports and usage by @jbischof in https://github.com/keras-team/keras-nlp/pull/262
Add BLEU Score by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/222
Configure GKE-based accelerator testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/265
Added WordPieceTokenizer training function by @jessechancy in https://github.com/keras-team/keras-nlp/pull/256
Add requirements.txt for cloud build by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/267
Global Seed Bug Fix by @jessechancy in https://github.com/keras-team/keras-nlp/pull/269
Update accelerator testing to use the new GCP project by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/272
Fixed typo: "recieved" by @ehrencrona in https://github.com/keras-team/keras-nlp/pull/273
Reuse dense pooled output for fine tuning by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/251
Simplify BERT modeling, use keras embeddings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/253
Rename UnicodeCharacterTokenizer>UnicodeCodepointTokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/254
Add README for accelerator testing config folder by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/276
Random Deletion Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/214
Made trainer more efficient. Loading full files instead of using TextLineDataset. by @jessechancy in https://github.com/keras-team/keras-nlp/pull/280
Use KerasNLP for BERT preprocessing for GLUE by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/252
Minor fixes to the Random Deletion Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/286
Fixes for WordPieceTrainer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/293
Update default to strip_accents=False by @jessechancy in https://github.com/keras-team/keras-nlp/pull/289
Move Bert to models folder by @jbischof in https://github.com/keras-team/keras-nlp/pull/288
Make Decoding Functions Graph-compatible (with XLA Support!) by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/271
SentencePieceTrainer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/281
Rename models.Bert() to models.BertCustom() by @jbischof in https://github.com/keras-team/keras-nlp/pull/310
Add a test for variable sequence length inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/313
Support checkpoint loading for BertBase by @jbischof in https://github.com/keras-team/keras-nlp/pull/299
RoBERTa pretrained model forward pass by @jessechancy in https://github.com/keras-team/keras-nlp/pull/304
Register objects as serializable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/292
Style merging for Bert and Roberta by @jbischof in https://github.com/keras-team/keras-nlp/pull/315
Streamline and speed up tests by @jbischof in https://github.com/keras-team/keras-nlp/pull/324
Add Support for CJK Char Splitting for WordPiece Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/318
Clean up model input names for consistency by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/327
Return a single tensor from roberta by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/328
BERT, RoBERTa: Add model.compile UTs by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/330
Continue rename of bert model inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/329
Text Generation Utilities: Add Support for Ragged Inputs by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/300
bert_base_zh, bert_base_multi_cased: Add BERT Base Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/319
WordPiece vocabularies trainer on Wikipedia dataset by @jessechancy in https://github.com/keras-team/keras-nlp/pull/316
Use the exported ragged ops for RandomDeletion by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/332
Random Swap Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/224
Fixes for Random Deletion Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/339
Move cloudbuild to a hidden directory by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/345
Fix the build by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/349
Migrating from Datasets to TFDS for GLUE Example by @aflah02 in https://github.com/keras-team/keras-nlp/pull/340
Move network_tests into keras_nlp/ by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/344
Stop hardcoding 2.9 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/351
Add BERT Large by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/331
Add normalize_first arg to Transformer Layers by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/350
Add Small BERT Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/338
Beam Search: Add Ragged and XLA Support by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/341
Fix download paths for bert weights by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/356
Add a BertPreprocessor class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/343
Text Generation Functions: Add Benchmark Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/342
Improve readability for encoder/decoder blocks by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/353
Add GPT-2 Model and its Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/354
Clean up BERT, RoBERTa doc-strings by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/359
Create unique string id for each BERT backbone by @jbischof in https://github.com/keras-team/keras-nlp/pull/361
Use model.fit() for BERT Example by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/360
Minor Fixes in BertPreprocessor Layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/373
Clone user passed initializers called multiple times by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/371
Update BERT model file structure by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/376
Move gpt model code into a directory by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/379
Move roberta model code into a directory by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/380
Reorg test directories by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/384
Add XLM-RoBERTa by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/372
Add DistilBERT by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/382
Stop running CI on Windows by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/386
Fix Bert serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/385
Improve MacOS support and pin tensorflow version during testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/383
Unify BERT model API in one class by @jbischof in https://github.com/keras-team/keras-nlp/pull/387
Add from_preset constructor to BertPreprocessor by @jbischof in https://github.com/keras-team/keras-nlp/pull/390
More robustly test BERT preprocessing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/394
Move name and trainable to kwargs by @jbischof in https://github.com/keras-team/keras-nlp/pull/399
Add backbone as property for task models by @jbischof in https://github.com/keras-team/keras-nlp/pull/398
Set default name of Bert instance to "backbone" by @jbischof in https://github.com/keras-team/keras-nlp/pull/397
Fix gpt2 serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/391
Fix distilbert serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/392
Fix roberta and xlm-roberta serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/393
Register the BertPreprocessor as serializable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/401
BPE tokenizer by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/389
Change GPT-2's Format to Mirror BERT's by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/418
Fix bert preprocessing docstring so it is runnable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/421
Change RoBERTa and XLM-RoBERTa's Format to Mirror BERT's by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/417
Update distilbert to mirror recent bert changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/406
Change gpt2 to GPT2 by @sampathweb in https://github.com/keras-team/keras-nlp/pull/425
Fix byte pair detokenization of 2d arrays by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/423
Never pass Raggeds to user function when generating text by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/424
Add XLMRobertaClassifier by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/422
Add RobertaPreprocessor Layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/419
Update Style Guide for naming of Models and Layers by @sampathweb in https://github.com/keras-team/keras-nlp/pull/434
Support String Output for BytePairTokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/438
Improve our continuous testing for model presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/357
Remove remote files from BPE docstring by @jbischof in https://github.com/keras-team/keras-nlp/pull/440
Add DistilBertClassifier by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/437
Remove lingering reference to BertCustom by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/441
Add XLM-RoBERTa Tokenizer (SPM) by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/428
Add a disclaimer for use of model checkpoints by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/430
Add a disclaimer to our README by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/431
Fix our BERT GLUE example so it runs again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/444
Add backbone presets to task classes by @jbischof in https://github.com/keras-team/keras-nlp/pull/448
Split the Bert tokenizer to a separate class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/449
Conditionally import tf text by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/452
Copy our model disclaimer to the distilbert classifier by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/453
Fix regex string for BPE by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/458
Fix docstrings and add note to style guide by @jbischof in https://github.com/keras-team/keras-nlp/pull/464
Allow formatting our docstrings inline by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/450
Update self.assertEquals with self.assertEqual by @MaximSmolskiy in https://github.com/keras-team/keras-nlp/pull/466
Document our release process by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/473
Add RobertaTokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/468
Add DistilBertTokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/469
Modify XLMRobertaTokenizer to Match BERT by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/471
Add GPT2Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/470
Minor fix to git commands by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/475
Version bump to 0.4.0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/476
Clarify comment on BERT preset testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/477
Fix the nightly build by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/484
Bump tf and tf-text to 2.11 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/490
Consolidate preset testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/480
Allow BertPreprocessor to map labeled datasets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/478
Glue eval script by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/445
Update Requirements and Python version in setup.py by @sampathweb in https://github.com/keras-team/keras-nlp/pull/495
First task-level preset with BertClassifier by @jbischof in https://github.com/keras-team/keras-nlp/pull/494
Add a helper model to automatically apply preprocessing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/346
Add GPT2 Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/472
fix incorrect flag by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/496
Add instructions on how to update deps of GPU testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/499
Add XLM-RoBERTa Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/482
Add DistilBERT Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/479
Add RoBERTa Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/506
Fix nightly builds by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/522
Remove typo by @jbischof in https://github.com/keras-team/keras-nlp/pull/515
Fix Model Doc-string Examples by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/516
Mark format.sh executable again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/518
Make BertClassifier operate directly on raw string inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/485
Fix nightlies take two by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/525
Use tf.ones for docstring example input by @jbischof in https://github.com/keras-team/keras-nlp/pull/524
Fix the index order of GLUE script and other bugs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/517
Standardize on "backbone" naming for BERT by @jbischof in https://github.com/keras-team/keras-nlp/pull/536
Add dropout to BertClassifier by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/540
Preprocess string lists as a batch of single segments by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/504
Update model and file names for DistilBert by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/541
Rename filenames in models/ to match classnames by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/548
Split tokenizers into their own file by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/549
Add a note about our new file naming conventions by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/553
Add distribution support for GLUE script by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/544
Make out preset ids more consistent by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/552
Rename DistilBert -> DistilBertBackbone by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/551
Rename Roberta -> RobertaBackbone (and same for XLM*) by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/550
Fix link in glue_benchmark README by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/557
Remove qualification for PRESET_NAMES by @jbischof in https://github.com/keras-team/keras-nlp/pull/554
Use black[jupyter] to format notebooks by @jbischof in https://github.com/keras-team/keras-nlp/pull/556
Temporarily drop GPT2 from our init.py by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/560
Replicate #536 changing GPT2 -> GPT2Backbone by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/558
Change Backbone Names by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/559
Raise a friendly error message for unbatched input by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/545
Fix Typo in Backbone Doc-strings by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/565
Fix LinearDecayWithWarmup crash in BERT model. by @reedwm in https://github.com/keras-team/keras-nlp/pull/564
Update XLMRobertaPreprocessor to mirror recent changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/568
Update RobertaPreprocessor to mirror recent changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/567
Fix minor typo in BertPreprocessor layer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/569
Update DistilBertPreprocessor to mirror recent changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/566
Point presets to url containing "v1/" by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/577
Stop testing h5 saved model format, start testing keras_v3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/521
fix the gpu testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/581
Make DistilBertClassifier operate directly on raw string inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/578
Make RobertaClassifier operate directly on raw string inputs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/579
Export XLM-Roberta Classifier by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/580
Make XLMRobertaClassifier operate directly on raw string inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/583
Fix some misc typos for distilbert by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/584
Report GLUE score and hyperparameter settings by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/585
Remove support for Python 3.7 to align with keras-nightly by @sampathweb in https://github.com/keras-team/keras-nlp/pull/590
Add DeBERTa v3 Model by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/435
Add DeBERTa Tokenizer and Preprocessor Classes by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/589
Add Dropout to *Classifier Doc-strings by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/595
Rename MLM -> MaskedLM for all library symbols by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/598
File-level Doc-string Changes for Classifiers and Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/604
Add DebertaClassifier and DeBERTa Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/594
Rename Deberta -> DebertaV3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/605
Version bump to 0.4.0.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/609
Add mixed precision support for glue script by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/608
Remove deberta from 0.4 release by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/607
Update README for v0.4 by @jbischof in https://github.com/keras-team/keras-nlp/pull/588
Remove the dev prefix for final release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/613
Rename preset IDs for consistency by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/612

New Contributors

@jbischof made their first contribution in https://github.com/keras-team/keras-nlp/pull/258
@ehrencrona made their first contribution in https://github.com/keras-team/keras-nlp/pull/273
@sampathweb made their first contribution in https://github.com/keras-team/keras-nlp/pull/425
@MaximSmolskiy made their first contribution in https://github.com/keras-team/keras-nlp/pull/466
@ADITYADAS1999 made their first contribution in https://github.com/keras-team/keras-nlp/pull/541
@reedwm made their first contribution in https://github.com/keras-team/keras-nlp/pull/564

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.4.0

keras-hub - v0.4.0.dev0

Published by mattdangerw almost 2 years ago

⚠️⚠️⚠️ This is a pre-release for testing purposes, documentation for this release has not yet shipped.

The KerasNLP 0.4 adds support for pretrained models to the API via keras_nlp.models. If you encounter any problems or have questions, please open an issue or discussion of the discussion tab!

Breaking Changes

Renamed keras_nlp.layers.MLMHead -> keras_nlp.layers.MaskedLMHead.
Renamed keras_nlp.layers.MLMMaskGenerator -> keras_nlp.layers.MaskedLMMaskGenerator.
Renamed keras_nlp.layers.UnicodeCharacterTokenizer -> keras_nlp.layers.UnicodeCodepointTokenizer.
Switched the default of lowercase in keras_nlp.tokenizers.WordPieceTokenizer from True to False.
Renamed the token id output of MaskedLMMaskGenerator from "tokens" to "tokens_ids".

Summary

Added the keras_nlp.models API.
- Adds support for BERT, DistilBERT, RoBERTa, and XLM-RoBERTa models and pretrained checkpoints.
Added new metrics.
- keras_nlp.metrics.Bleu and keras_nlp.metrics.EditDistance.
Added new vocabulary training utilities.
- keras_nlp.tokenizers.compute_word_piece_vocabulary and keras_nlp.tokenizers.compute_sentence_piece_proto.

What's Changed

Add Edit Distance Metric by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/231
Minor fix to simplify and test handling of max_length prompts by @jbischof in https://github.com/keras-team/keras-nlp/pull/258
Remove split regex args for WordPieceTokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/255
Add instructions on installing the latest changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/261
Add warning when k > vocab_size in top_k_search by @jbischof in https://github.com/keras-team/keras-nlp/pull/260
Fix keras library imports and usage by @jbischof in https://github.com/keras-team/keras-nlp/pull/262
Add BLEU Score by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/222
Configure GKE-based accelerator testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/265
Added WordPieceTokenizer training function by @jessechancy in https://github.com/keras-team/keras-nlp/pull/256
Add requirements.txt for cloud build by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/267
Global Seed Bug Fix by @jessechancy in https://github.com/keras-team/keras-nlp/pull/269
Update accelerator testing to use the new GCP project by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/272
Fixed typo: "recieved" by @ehrencrona in https://github.com/keras-team/keras-nlp/pull/273
Reuse dense pooled output for fine tuning by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/251
Simplify BERT modeling, use keras embeddings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/253
Rename UnicodeCharacterTokenizer>UnicodeCodepointTokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/254
Add README for accelerator testing config folder by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/276
Random Deletion Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/214
Made trainer more efficient. Loading full files instead of using TextLineDataset. by @jessechancy in https://github.com/keras-team/keras-nlp/pull/280
Use KerasNLP for BERT preprocessing for GLUE by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/252
Minor fixes to the Random Deletion Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/286
Fixes for WordPieceTrainer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/293
Update default to strip_accents=False by @jessechancy in https://github.com/keras-team/keras-nlp/pull/289
Move Bert to models folder by @jbischof in https://github.com/keras-team/keras-nlp/pull/288
Make Decoding Functions Graph-compatible (with XLA Support!) by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/271
SentencePieceTrainer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/281
Rename models.Bert() to models.BertCustom() by @jbischof in https://github.com/keras-team/keras-nlp/pull/310
Add a test for variable sequence length inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/313
Support checkpoint loading for BertBase by @jbischof in https://github.com/keras-team/keras-nlp/pull/299
RoBERTa pretrained model forward pass by @jessechancy in https://github.com/keras-team/keras-nlp/pull/304
Register objects as serializable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/292
Style merging for Bert and Roberta by @jbischof in https://github.com/keras-team/keras-nlp/pull/315
Streamline and speed up tests by @jbischof in https://github.com/keras-team/keras-nlp/pull/324
Add Support for CJK Char Splitting for WordPiece Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/318
Clean up model input names for consistency by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/327
Return a single tensor from roberta by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/328
BERT, RoBERTa: Add model.compile UTs by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/330
Continue rename of bert model inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/329
Text Generation Utilities: Add Support for Ragged Inputs by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/300
bert_base_zh, bert_base_multi_cased: Add BERT Base Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/319
WordPiece vocabularies trainer on Wikipedia dataset by @jessechancy in https://github.com/keras-team/keras-nlp/pull/316
Use the exported ragged ops for RandomDeletion by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/332
Random Swap Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/224
Fixes for Random Deletion Layer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/339
Move cloudbuild to a hidden directory by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/345
Fix the build by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/349
Migrating from Datasets to TFDS for GLUE Example by @aflah02 in https://github.com/keras-team/keras-nlp/pull/340
Move network_tests into keras_nlp/ by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/344
Stop hardcoding 2.9 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/351
Add BERT Large by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/331
Add normalize_first arg to Transformer Layers by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/350
Add Small BERT Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/338
Beam Search: Add Ragged and XLA Support by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/341
Fix download paths for bert weights by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/356
Add a BertPreprocessor class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/343
Text Generation Functions: Add Benchmark Script by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/342
Improve readability for encoder/decoder blocks by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/353
Add GPT-2 Model and its Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/354
Clean up BERT, RoBERTa doc-strings by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/359
Create unique string id for each BERT backbone by @jbischof in https://github.com/keras-team/keras-nlp/pull/361
Use model.fit() for BERT Example by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/360
Minor Fixes in BertPreprocessor Layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/373
Clone user passed initializers called multiple times by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/371
Update BERT model file structure by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/376
Move gpt model code into a directory by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/379
Move roberta model code into a directory by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/380
Reorg test directories by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/384
Add XLM-RoBERTa by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/372
Add DistilBERT by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/382
Stop running CI on Windows by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/386
Fix Bert serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/385
Improve MacOS support and pin tensorflow version during testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/383
Unify BERT model API in one class by @jbischof in https://github.com/keras-team/keras-nlp/pull/387
Add from_preset constructor to BertPreprocessor by @jbischof in https://github.com/keras-team/keras-nlp/pull/390
More robustly test BERT preprocessing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/394
Move name and trainable to kwargs by @jbischof in https://github.com/keras-team/keras-nlp/pull/399
Add backbone as property for task models by @jbischof in https://github.com/keras-team/keras-nlp/pull/398
Set default name of Bert instance to "backbone" by @jbischof in https://github.com/keras-team/keras-nlp/pull/397
Fix gpt2 serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/391
Fix distilbert serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/392
Fix roberta and xlm-roberta serialization by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/393
Register the BertPreprocessor as serializable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/401
BPE tokenizer by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/389
Change GPT-2's Format to Mirror BERT's by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/418
Fix bert preprocessing docstring so it is runnable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/421
Change RoBERTa and XLM-RoBERTa's Format to Mirror BERT's by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/417
Update distilbert to mirror recent bert changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/406
Change gpt2 to GPT2 by @sampathweb in https://github.com/keras-team/keras-nlp/pull/425
Fix byte pair detokenization of 2d arrays by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/423
Never pass Raggeds to user function when generating text by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/424
Add XLMRobertaClassifier by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/422
Add RobertaPreprocessor Layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/419
Update Style Guide for naming of Models and Layers by @sampathweb in https://github.com/keras-team/keras-nlp/pull/434
Support String Output for BytePairTokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/438
Improve our continuous testing for model presets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/357
Remove remote files from BPE docstring by @jbischof in https://github.com/keras-team/keras-nlp/pull/440
Add DistilBertClassifier by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/437
Remove lingering reference to BertCustom by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/441
Add XLM-RoBERTa Tokenizer (SPM) by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/428
Add a disclaimer for use of model checkpoints by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/430
Add a disclaimer to our README by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/431
Fix our BERT GLUE example so it runs again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/444
Add backbone presets to task classes by @jbischof in https://github.com/keras-team/keras-nlp/pull/448
Split the Bert tokenizer to a separate class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/449
Conditionally import tf text by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/452
Copy our model disclaimer to the distilbert classifier by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/453
Fix regex string for BPE by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/458
Fix docstrings and add note to style guide by @jbischof in https://github.com/keras-team/keras-nlp/pull/464
Allow formatting our docstrings inline by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/450
Update self.assertEquals with self.assertEqual by @MaximSmolskiy in https://github.com/keras-team/keras-nlp/pull/466
Document our release process by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/473
Add RobertaTokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/468
Add DistilBertTokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/469
Modify XLMRobertaTokenizer to Match BERT by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/471
Add GPT2Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/470
Minor fix to git commands by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/475
Version bump to 0.4.0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/476
Clarify comment on BERT preset testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/477
Fix the nightly build by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/484
Bump tf and tf-text to 2.11 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/490
Consolidate preset testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/480
Allow BertPreprocessor to map labeled datasets by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/478
Glue eval script by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/445
Update Requirements and Python version in setup.py by @sampathweb in https://github.com/keras-team/keras-nlp/pull/495
First task-level preset with BertClassifier by @jbischof in https://github.com/keras-team/keras-nlp/pull/494
Add a helper model to automatically apply preprocessing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/346
Add GPT2 Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/472
fix incorrect flag by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/496
Add instructions on how to update deps of GPU testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/499
Add XLM-RoBERTa Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/482
Add DistilBERT Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/479
Add RoBERTa Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/506
Fix nightly builds by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/522
Remove typo by @jbischof in https://github.com/keras-team/keras-nlp/pull/515
Fix Model Doc-string Examples by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/516
Mark format.sh executable again by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/518
Make BertClassifier operate directly on raw string inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/485
Fix nightlies take two by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/525
Use tf.ones for docstring example input by @jbischof in https://github.com/keras-team/keras-nlp/pull/524
Fix the index order of GLUE script and other bugs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/517
Standardize on "backbone" naming for BERT by @jbischof in https://github.com/keras-team/keras-nlp/pull/536
Add dropout to BertClassifier by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/540
Preprocess string lists as a batch of single segments by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/504
Update model and file names for DistilBert by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/541
Rename filenames in models/ to match classnames by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/548
Split tokenizers into their own file by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/549
Add a note about our new file naming conventions by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/553
Add distribution support for GLUE script by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/544
Make out preset ids more consistent by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/552
Rename DistilBert -> DistilBertBackbone by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/551
Rename Roberta -> RobertaBackbone (and same for XLM*) by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/550
Fix link in glue_benchmark README by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/557
Remove qualification for PRESET_NAMES by @jbischof in https://github.com/keras-team/keras-nlp/pull/554
Use black[jupyter] to format notebooks by @jbischof in https://github.com/keras-team/keras-nlp/pull/556
Temporarily drop GPT2 from our init.py by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/560
Replicate #536 changing GPT2 -> GPT2Backbone by @ADITYADAS1999 in https://github.com/keras-team/keras-nlp/pull/558
Change Backbone Names by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/559
Raise a friendly error message for unbatched input by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/545
Fix Typo in Backbone Doc-strings by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/565
Fix LinearDecayWithWarmup crash in BERT model. by @reedwm in https://github.com/keras-team/keras-nlp/pull/564
Update XLMRobertaPreprocessor to mirror recent changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/568
Update RobertaPreprocessor to mirror recent changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/567
Fix minor typo in BertPreprocessor layer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/569
Update DistilBertPreprocessor to mirror recent changes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/566
Point presets to url containing "v1/" by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/577
Stop testing h5 saved model format, start testing keras_v3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/521
fix the gpu testing by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/581
Make DistilBertClassifier operate directly on raw string inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/578
Make RobertaClassifier operate directly on raw string inputs by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/579
Export XLM-Roberta Classifier by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/580
Make XLMRobertaClassifier operate directly on raw string inputs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/583
Fix some misc typos for distilbert by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/584
Report GLUE score and hyperparameter settings by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/585
Remove support for Python 3.7 to align with keras-nightly by @sampathweb in https://github.com/keras-team/keras-nlp/pull/590
Add DeBERTa v3 Model by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/435
Add DeBERTa Tokenizer and Preprocessor Classes by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/589
Add Dropout to *Classifier Doc-strings by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/595
Rename MLM -> MaskedLM for all library symbols by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/598
File-level Doc-string Changes for Classifiers and Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/604
Add DebertaClassifier and DeBERTa Presets by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/594
Rename Deberta -> DebertaV3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/605
Version bump to 0.4.0.dev0 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/609

New Contributors

@jbischof made their first contribution in https://github.com/keras-team/keras-nlp/pull/258
@ehrencrona made their first contribution in https://github.com/keras-team/keras-nlp/pull/273
@sampathweb made their first contribution in https://github.com/keras-team/keras-nlp/pull/425
@MaximSmolskiy made their first contribution in https://github.com/keras-team/keras-nlp/pull/466
@ADITYADAS1999 made their first contribution in https://github.com/keras-team/keras-nlp/pull/541
@reedwm made their first contribution in https://github.com/keras-team/keras-nlp/pull/564

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.4.0.dev0

keras-hub - v0.3.1

Published by mattdangerw almost 2 years ago

Summary

Add keras_nlp.tokenizers.BytePairTokenizer with tf.data friendly support for the tokenization used by GPT-2, RoBERTa and other models.
Remove the hard dependency on tensorflow and tensorflow-text when pip installing on MacOS, to accommodate M1 chips. See this section of our contributor guide for more information on MacOS development.

What's Changed

Cherry picks 0.3 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/454
Bump version for 0.3.1 pre release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/456
Remove dev prefix for 0.3.1 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/457

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.3.1

keras-hub - v0.3.0

Published by mattdangerw over 2 years ago

Summary

Added keras_nlp.tokenizers.SentencePieceTokenizer.
Added two token packing layers keras_nlp.layers.StartEndPacker and keras_nlp.layers.MultiSegmentPacker.
Added two metrics, keras_nlp.metrics.RougeL and keras_nlp.metrics.RougeN based on the rouge-score package.
Added five utilities for generating sequences, keras_nlp.utils.greedy_search, keras_nlp.utils.random_search, keras_nlp.utils.top_k_search, keras_nlp.utils.top_p_search, keras_nlp.utils.beam_search.

What's Changed

Greedy text generation util by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/154
Remove incorrect embedding size limit by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/195
Fix inits for bert heads by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/192
Add keras.io links to README by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/196
Minor Corrections In ROADMAP.md by @saiteja13427 in https://github.com/keras-team/keras-nlp/pull/200
Fix Loose Dependency Imports by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/199
Reorganize examples by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/179
Remove bert config arguments from README by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/205
Add checkpoints to BERT training by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/184
Run keras tuner from a temp directory by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/202
Token and position embedding minor fixes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/203
Correct typo in WordPieceTokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/208
Add TPU support to BERT example by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/207
Remove type annotations for complex types by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/194
Issue 182: Modified TransformerDecoder with optional parameter by @jessechancy in https://github.com/keras-team/keras-nlp/pull/217
Add StartEndPacker layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/221
Add a layer for packing inputs for BERT-likes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/88
Ignore UserWarning to fix nightly testing breakage by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/227
Add ROUGE Metric by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/122
Allow long lines for links in docstrings by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/229
Random Sampling Util for Text Generation by @jessechancy in https://github.com/keras-team/keras-nlp/pull/228
added top k search util by @jessechancy in https://github.com/keras-team/keras-nlp/pull/232
top p search and testing by @jessechancy in https://github.com/keras-team/keras-nlp/pull/233
Add a SentencePiece tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/218
Add cloud training support for BERT example by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/226
Bump version to 0.3.0 for upcoming release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/239
Add support for StartEndPacker packing 2D tensor by @jessechancy in https://github.com/keras-team/keras-nlp/pull/240
Fixed Bug with Unicode Tokenizer Vocab Size by @aflah02 in https://github.com/keras-team/keras-nlp/pull/243
Fixed Import for top_p_search util by @aflah02 in https://github.com/keras-team/keras-nlp/pull/245
MultiSegmentPacker support for 2D dense tensor by @jessechancy in https://github.com/keras-team/keras-nlp/pull/244
Minor fixes for multi-segment packer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/246
Add beam search decoding util by @jessechancy in https://github.com/keras-team/keras-nlp/pull/237

New Contributors

@saiteja13427 made their first contribution in https://github.com/keras-team/keras-nlp/pull/200
@jessechancy made their first contribution in https://github.com/keras-team/keras-nlp/pull/217

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.2.0...v0.3.0

keras-hub - v0.2.0

Published by mattdangerw over 2 years ago

Summary

Documentation live on keras.io.
Added two tokenizers: ByteTokenizer and UnicodeCharacterTokenizer.
Added a Perplexity metric.
Added three layers TokenAndPositionEmbedding, MLMMaskGenerator and MLMHead.
Contributing guides and roadmap.

What's Changed

Add Byte Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/80
Fixing rank 1 outputs for WordPieceTokenizer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/92
Add tokenizer accessors to the base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/89
Fix word piece attributes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/97
Small fix: change assertEquals to assertEqual by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/103
Added a Learning Rate Schedule for the BERT Example by @Stealth-py in https://github.com/keras-team/keras-nlp/pull/96
Add Perplexity Metric by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/68
Use the black profile for isort by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/117
Update README with release information by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/118
Add a class to generate LM masks by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/61
Add docstring testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/116
Fix broken docstring in MLMMaskGenerator by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/121
Adding a UnicodeCharacterTokenizer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/100
Added Class by @adhadse in https://github.com/keras-team/keras-nlp/pull/91
Fix bert example so it is runnable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/123
Fix the issue that MLMMaskGenerator does not work in graph mode by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/131
Actually use layer norm epsilon in encoder/decoder by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/133
Whitelisted formatting and lint check targets by @adhadse in https://github.com/keras-team/keras-nlp/pull/126
Updated CONTRIBUTING.md for setup of venv and standard pip install by @adhadse in https://github.com/keras-team/keras-nlp/pull/127
Fix mask propagation of transformer layers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/139
Fix masking for TokenAndPositionEmbedding by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/140
Fixed no oov token error in vocab for WordPieceTokenizer by @adhadse in https://github.com/keras-team/keras-nlp/pull/136
Add a MLMHead layer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/132
Bump version for 0.2.0 dev release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/142
Added WSL setup text to CONTRIBUTING.md by @adhadse in https://github.com/keras-team/keras-nlp/pull/144
Add attribution for the BERT modeling code by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/151
Remove preprocessing subdir by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/150
Word piece arg change by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/148
Rename max_length to sequence_length by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/149
Don't accept a string dtype for unicode tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/147
Adding Utility to Detokenize as list of Strings to Tokenizer Base Class by @aflah02 in https://github.com/keras-team/keras-nlp/pull/124
Fixed Import Error by @aflah02 in https://github.com/keras-team/keras-nlp/pull/161
Added KerasTuner Hyper-Parameter Search for the BERT fine-tuning script. by @Stealth-py in https://github.com/keras-team/keras-nlp/pull/143
Docstring updates for upcoming doc publish by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/146
version bump for 0.2.0.dev2 pre-release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/165
Added a vocabulary_size argument to UnicodeCharacterTokenizer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/163
Simplified utility to preview a tfrecord by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/168
Update BERT example's README with data downloading instructions by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/169
Add a call to repeat during pretraining by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/172
Add an integration test matching our quick start by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/162
Modify README of bert example by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/174
Fix the finetuning script's loss and metric config by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/176
Minor improvements to the position embedding docs by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/180
Update docs for upcoming 0.2.0 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/158
Restore accidentally deleted line from README by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/185
Bump version for 0.2.0 release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/186
Pre release fix by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/187

New Contributors

@Stealth-py made their first contribution in https://github.com/keras-team/keras-nlp/pull/96
@adhadse made their first contribution in https://github.com/keras-team/keras-nlp/pull/91

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.1.1...v0.2.0

keras-hub - v0.2.0.dev2

Published by mattdangerw over 2 years ago

What's Changed

Added WSL setup text to CONTRIBUTING.md by @adhadse in https://github.com/keras-team/keras-nlp/pull/144
Add attribution for the BERT modeling code by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/151
Remove preprocessing subdir by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/150
Word piece arg change by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/148
Rename max_length to sequence_length by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/149
Don't accept a string dtype for unicode tokenizer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/147
Adding Utility to Detokenize as list of Strings to Tokenizer Base Class by @aflah02 in https://github.com/keras-team/keras-nlp/pull/124
Fixed Import Error by @aflah02 in https://github.com/keras-team/keras-nlp/pull/161
Added KerasTuner Hyper-Parameter Search for the BERT fine-tuning script. by @Stealth-py in https://github.com/keras-team/keras-nlp/pull/143
Docstring updates for upcoming doc publish by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/146
version bump for 0.2.0.dev2 pre-release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/165

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.2.0-dev.1...v0.2.0.dev2

keras-hub - v0.2.0-dev.1

Published by mattdangerw over 2 years ago

What's Changed

Add Byte Tokenizer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/80
Fixing rank 1 outputs for WordPieceTokenizer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/92
Add tokenizer accessors to the base class by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/89
Fix word piece attributes by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/97
Small fix: change assertEquals to assertEqual by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/103
Added a Learning Rate Schedule for the BERT Example by @Stealth-py in https://github.com/keras-team/keras-nlp/pull/96
Add Perplexity Metric by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/68
Use the black profile for isort by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/117
Update README with release information by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/118
Add a class to generate LM masks by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/61
Add docstring testing by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/116
Fix broken docstring in MLMMaskGenerator by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/121
Adding a UnicodeCharacterTokenizer by @aflah02 in https://github.com/keras-team/keras-nlp/pull/100
Added TokenAndPositionEmbedding Class by @adhadse in https://github.com/keras-team/keras-nlp/pull/91
Fix bert example so it is runnable by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/123
Fix the issue that MLMMaskGenerator does not work in graph mode by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/131
Actually use layer norm epsilon in encoder/decoder by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/133
Whitelisted formatting and lint check targets by @adhadse in https://github.com/keras-team/keras-nlp/pull/126
Updated CONTRIBUTING.md for setup of venv and standard pip install by @adhadse in https://github.com/keras-team/keras-nlp/pull/127
Fix mask propagation of transformer layers by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/139
Fix masking for TokenAndPositionEmbedding by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/140
Fixed no oov token error in vocab for WordPieceTokenizer by @adhadse in https://github.com/keras-team/keras-nlp/pull/136
Add a MLMHead layer by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/132
Bump version for 0.2.0 dev release by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/142

New Contributors

@Stealth-py made their first contribution in https://github.com/keras-team/keras-nlp/pull/96
@adhadse made their first contribution in https://github.com/keras-team/keras-nlp/pull/91

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.1.1...v0.2.0-dev.1

keras-hub - v0.1.1

Published by mattdangerw over 2 years ago

What's Changed

Add tokenizer helper to convert tokens to ids by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/75
Add a sinusoidal embedding layer by @amantayal44 in https://github.com/keras-team/keras-nlp/pull/59
Add a learned positional embedding layer by @hertschuh in https://github.com/keras-team/keras-nlp/pull/47
Fix typo in position embedding docstring by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/86
Bump version number to 0.1.1 by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/90

New Contributors

@amantayal44 made their first contribution in https://github.com/keras-team/keras-nlp/pull/59
@hertschuh made their first contribution in https://github.com/keras-team/keras-nlp/pull/47

Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.1.0...v0.1.1

keras-hub -

Published by mattdangerw over 2 years ago

Initial release of keras-nlp with word piece tokenizer and transformer encoder/decoder blocks.

This is a v0 release, with no API compatibility guarantees.

Package Rankings

Top 34.6% on Pypi.org

Badges

Extracted from project README

Related Projects

tf-keras

The TensorFlow-specific implementation of the Keras API, which was the default Keras from 2019 to...

03 Sep 2023 60

autokeras

AutoML library for deep learning

19 Nov 2017 9,102

hub

A library for transfer learning by reusing parts of TensorFlow models.

12 Mar 2018 3,434

keras-tuner

A Hyperparameter Tuning Library for Keras

06 Jun 2019 2,831

keras-aug

A library that includes pure TF/Keras preprocessing and augmentation layers, providing support fo...

10 Apr 2023 8

DLTK

Deep Learning Toolkit for Medical Image Analysis

02 May 2017 1,426

keras-text

Text Classification Library in Keras

27 Aug 2017 420

Open3D-ML

An extension of Open3D to address 3D Machine Learning tasks

01 Jul 2020 1,824

keras

Deep Learning for humans

28 Mar 2015 60,873

ktrain

ktrain is a Python library that makes deep learning and AI more accessible and easier to apply

06 Feb 2019 1,226

MMdnn

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g...

16 Aug 2017 5,788