Modular Natural Language Processing workflows with Keras
APACHE-2.0 License
Bot releases are hidden (Show)
Published by mattdangerw over 1 year ago
KerasNLP is adding experimental support for Jax and PyTorch backends on top of the Keras Core library. Read the anouncement, and browse the full library documentation, including how to specify the backend when running your code.
Support for both Jax and PyTorch is still experimental, expect some rough edges and please give us feedback!
0.5.2
with the addition of multi-backend support.keras_nlp.models.DebertaV3*
keras_nlp.models.FNet*
keras_nlp.metrics
keras_nlp.samplers.BeamSampler
keras_nlp.samplers.ContrastiveSampler
KERAS_BACKEND={jax, torch, tensorflow}
, you will be trying the new Keras Core library, using the specified backend. This is a great way to test out the future of the library!Defaults to
to end of arg docstring and standardise values by @SamuelMarks in https://github.com/keras-team/keras-nlp/pull/1057
GPTNeoXBackbone
by @shivance in https://github.com/keras-team/keras-nlp/pull/1056
GPTNeoXPreprocessor
by @shivance in https://github.com/keras-team/keras-nlp/pull/1093
GPTNeoXCausalLMPreprocessor
by @shivance in https://github.com/keras-team/keras-nlp/pull/1106
RotaryEmbedding
and GPTNeoXAttention
by @shivance in https://github.com/keras-team/keras-nlp/pull/1101
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.5.2...v0.6.0
Published by mattdangerw over 1 year ago
🚧 Dev release. 🚧
KerasNLP is adding experimental support for Jax and PyTorch backends on top of the Keras Core library. Read the anouncement, and browse the full library documentation, including how to specify the backend when running your code.
Support for both Jax and PyTorch is still experimental, expect some rough edges and please give us feedback!
0.5.2
with the addition of multi-backend support.keras_nlp.models.DebertaV3*
keras_nlp.models.FNet*
keras_nlp.metrics
keras_nlp.samplers.BeamSampler
keras_nlp.samplers.ContrastiveSampler
KERAS_BACKEND=tensorflow
, you will be trying the new Keras Core library, using the tensorflow
backend. This is a great way to test out the future of the library.self
in calls to super()
by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Backbone
base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
value_dim
in TransformerDecoder
's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
token_embedding
as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
from_preset
to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
AlbertClassifier
by @shivance in https://github.com/keras-team/keras-nlp/pull/668
GPT2Preprocessor
by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
_backbone
, _tokenizer
and _preprocessor
in Task by @jbischof in https://github.com/keras-team/keras-nlp/pull/899
sampler
arg by @jbischof in https://github.com/keras-team/keras-nlp/pull/964
generate()
by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/983
OPTCausalLM
and preprocessors by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/990
Defaults to
to end of arg docstring and standardise values by @SamuelMarks in https://github.com/keras-team/keras-nlp/pull/1057
GPTNeoXBackbone
by @shivance in https://github.com/keras-team/keras-nlp/pull/1056
GPTNeoXPreprocessor
by @shivance in https://github.com/keras-team/keras-nlp/pull/1093
GPTNeoXCausalLMPreprocessor
by @shivance in https://github.com/keras-team/keras-nlp/pull/1106
RotaryEmbedding
and GPTNeoXAttention
by @shivance in https://github.com/keras-team/keras-nlp/pull/1101
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.6.0.dev0
Published by chenmoneygithub over 1 year ago
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.5.1...v0.5.2
Published by chenmoneygithub over 1 year ago
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.5.0...v0.5.1
Published by chenmoneygithub over 1 year ago
In this 0.5 release, we are bringing generative AI to KerasNLP!
keras_nlp.models.GPT2CausalLM
and keras_nlp.models.OPTCausalLM
along with corresponding preprocessors. Both task models exposed a public generate()
method for text generation.keras_nlp.samplers
for better UX and scalability.keras_nlp.models.XXXMaskedLM
, e.g., keras_nlp.models.BertMaskedLM
.self
in calls to super()
by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Backbone
base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
value_dim
in TransformerDecoder
's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
token_embedding
as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
from_preset
to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
AlbertClassifier
by @shivance in https://github.com/keras-team/keras-nlp/pull/668
GPT2Preprocessor
by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
_backbone
, _tokenizer
and _preprocessor
in Task by @jbischof in https://github.com/keras-team/keras-nlp/pull/899
sampler
arg by @jbischof in https://github.com/keras-team/keras-nlp/pull/964
generate()
by @mattdangerw in https://github.com/keras-team/keras-nlp/pull/983
OPTCausalLM
and preprocessors by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/990
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.5.0
Published by mattdangerw over 1 year ago
The 0.4.1 release is a minor release with new model architectures and compilation defaults for task models. If you encounter any problems or have questions, please open an issue!
keras_nlp.models.BertClassifier
). No existing functionality is changed, but users of task models can now skip calling .compile()
and use default learning rates and optimization strategies provided by the library.keras_nlp.models.AlbertBackbone
, keras_nlp.models.AlbertClassifier
, preprocessor, and tokenizer layers for pre-trained ALBERT models.keras_nlp.models.FNetBackbone
, keras_nlp.models.FNetClassifier
, preprocessor, and tokenizer layers for pre-trained FNet models.keras_nlp.models.DebertaV3Backbone
, keras_nlp.models.DebertaV3Classifier
, preprocessor, and tokenizer layers for pre-trained DeBERTaV3 models.self
in calls to super()
by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Backbone
base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
value_dim
in TransformerDecoder
's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
token_embedding
as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
from_preset
to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
AlbertClassifier
by @shivance in https://github.com/keras-team/keras-nlp/pull/668
GPT2Preprocessor
by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.4.1
Published by mattdangerw over 1 year ago
self
in calls to super()
by @mbrukman in https://github.com/keras-team/keras-nlp/pull/628
Backbone
base class by @jbischof in https://github.com/keras-team/keras-nlp/pull/621
value_dim
in TransformerDecoder
's cross-attn layer by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/667
token_embedding
as a Backbone Property by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/676
from_preset
to base tokenizer classes by @shivance in https://github.com/keras-team/keras-nlp/pull/673
AlbertClassifier
by @shivance in https://github.com/keras-team/keras-nlp/pull/668
GPT2Preprocessor
by @chenmoneygithub in https://github.com/keras-team/keras-nlp/pull/704
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.4.0...v0.4.1.dev0
Published by mattdangerw almost 2 years ago
The 0.4 release adds support for pretrained models to the library via keras_nlp.models
. You can read an
introduction to the new API in our Getting Started Guide.
If you encounter any problems or have questions, please open an issue!
keras_nlp.layers.MLMHead
-> keras_nlp.layers.MaskedLMHead
.keras_nlp.layers.MLMMaskGenerator
-> keras_nlp.layers.MaskedLMMaskGenerator
.keras_nlp.layers.UnicodeCharacterTokenizer
-> keras_nlp.layers.UnicodeCodepointTokenizer
.lowercase
in keras_nlp.tokenizers.WordPieceTokenizer
from True
to False
.MaskedLMMaskGenerator
from "tokens"
to "tokens_ids"
.keras_nlp.models
API.
keras_nlp.metrics.Bleu
and keras_nlp.metrics.EditDistance
.keras_nlp.tokenizers.compute_word_piece_vocabulary
and keras_nlp.tokenizers.compute_sentence_piece_proto
.keras_nlp.layers.RandomSwap
and keras_nlp.layers.RandomDeletion
.models.Bert()
to models.BertCustom()
by @jbischof in https://github.com/keras-team/keras-nlp/pull/310
BertBase
by @jbischof in https://github.com/keras-team/keras-nlp/pull/299
model.compile
UTs by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/330
bert_base_zh
, bert_base_multi_cased
: Add BERT Base Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/319
from_preset
constructor to BertPreprocessor
by @jbischof in https://github.com/keras-team/keras-nlp/pull/390
name
and trainable
to kwargs
by @jbischof in https://github.com/keras-team/keras-nlp/pull/399
backbone
as property
for task models by @jbischof in https://github.com/keras-team/keras-nlp/pull/398
Bert
instance to "backbone"
by @jbischof in https://github.com/keras-team/keras-nlp/pull/397
BertClassifier
by @jbischof in https://github.com/keras-team/keras-nlp/pull/494
tf.ones
for docstring example input by @jbischof in https://github.com/keras-team/keras-nlp/pull/524
PRESET_NAMES
by @jbischof in https://github.com/keras-team/keras-nlp/pull/554
black[jupyter]
to format notebooks by @jbischof in https://github.com/keras-team/keras-nlp/pull/556
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.4.0
Published by mattdangerw almost 2 years ago
⚠️⚠️⚠️ This is a pre-release for testing purposes, documentation for this release has not yet shipped.
The KerasNLP 0.4 adds support for pretrained models to the API via keras_nlp.models
. If you encounter any problems or have questions, please open an issue or discussion of the discussion tab!
keras_nlp.layers.MLMHead
-> keras_nlp.layers.MaskedLMHead
.keras_nlp.layers.MLMMaskGenerator
-> keras_nlp.layers.MaskedLMMaskGenerator
.keras_nlp.layers.UnicodeCharacterTokenizer
-> keras_nlp.layers.UnicodeCodepointTokenizer
.lowercase
in keras_nlp.tokenizers.WordPieceTokenizer
from True
to False
.MaskedLMMaskGenerator
from "tokens"
to "tokens_ids"
.keras_nlp.models
API.
keras_nlp.metrics.Bleu
and keras_nlp.metrics.EditDistance
.keras_nlp.tokenizers.compute_word_piece_vocabulary
and keras_nlp.tokenizers.compute_sentence_piece_proto
.models.Bert()
to models.BertCustom()
by @jbischof in https://github.com/keras-team/keras-nlp/pull/310
BertBase
by @jbischof in https://github.com/keras-team/keras-nlp/pull/299
model.compile
UTs by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/330
bert_base_zh
, bert_base_multi_cased
: Add BERT Base Variants by @abheesht17 in https://github.com/keras-team/keras-nlp/pull/319
from_preset
constructor to BertPreprocessor
by @jbischof in https://github.com/keras-team/keras-nlp/pull/390
name
and trainable
to kwargs
by @jbischof in https://github.com/keras-team/keras-nlp/pull/399
backbone
as property
for task models by @jbischof in https://github.com/keras-team/keras-nlp/pull/398
Bert
instance to "backbone"
by @jbischof in https://github.com/keras-team/keras-nlp/pull/397
BertClassifier
by @jbischof in https://github.com/keras-team/keras-nlp/pull/494
tf.ones
for docstring example input by @jbischof in https://github.com/keras-team/keras-nlp/pull/524
PRESET_NAMES
by @jbischof in https://github.com/keras-team/keras-nlp/pull/554
black[jupyter]
to format notebooks by @jbischof in https://github.com/keras-team/keras-nlp/pull/556
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.4.0.dev0
Published by mattdangerw almost 2 years ago
keras_nlp.tokenizers.BytePairTokenizer
with tf.data
friendly support for the tokenization used by GPT-2, RoBERTa and other models.tensorflow
and tensorflow-text
when pip installing on MacOS, to accommodate M1 chips. See this section of our contributor guide for more information on MacOS development.Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.3.0...v0.3.1
Published by mattdangerw over 2 years ago
keras_nlp.tokenizers.SentencePieceTokenizer
.keras_nlp.layers.StartEndPacker
and keras_nlp.layers.MultiSegmentPacker
.keras_nlp.metrics.RougeL
and keras_nlp.metrics.RougeN
based on the rouge-score
package.keras_nlp.utils.greedy_search
, keras_nlp.utils.random_search
, keras_nlp.utils.top_k_search
, keras_nlp.utils.top_p_search
, keras_nlp.utils.beam_search
.Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.2.0...v0.3.0
Published by mattdangerw over 2 years ago
ByteTokenizer
and UnicodeCharacterTokenizer
.Perplexity
metric.TokenAndPositionEmbedding
, MLMMaskGenerator
and MLMHead
.Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.1.1...v0.2.0
Published by mattdangerw over 2 years ago
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.2.0-dev.1...v0.2.0.dev2
Published by mattdangerw over 2 years ago
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.1.1...v0.2.0-dev.1
Published by mattdangerw over 2 years ago
Full Changelog: https://github.com/keras-team/keras-nlp/compare/v0.1.0...v0.1.1