Improvements

• Added two NER DeBERTa models (#1691).

Improvements

• Added Python 3.11 support (#1681).
• Added NER model with 37 entities (#1682).

Improvements

• Added model usage examples to Quick Start (#1669).

Bug Fixes and Other Changes

• Fixed documentation building (#1673).
• Fixed wrong link in documentation, platforms list and multitask model evaluation (#1668, #1676, #1672).

Major Features and Improvements

• Models documentation brought to a uniform format: 1. Introduction to the task. 2. Get started with the model. 3. Models list. 4. Use the model for prediction. 5. Customize the model. rst docs converted in ipynb format. (#1644).

Bug Fixes and Other Changes

• Updated requirements (#1662, #1665).
• Fixed error when using multiple GPUs for NER models (#1661).
• Fixed issue #1642 (#1661).

Breaking Changes

• Removed model_name parameter from TorchModel and its derived classes. Now model should be initialized in component's __init__ method (#1617).
• Removed deeppavlov.core.models.lr_scheduled_model (#1617).
• TorchModel.__init__ now has one mandatory parameter - model (#1617).

Major Features and Improvements

• Few-Shot Text Classification Model (#1630).
• GLUE and SuperGLUE models major update (#1647).
• New components and configs for syntax parser and morphotagger (#1641).
• Improved quality and reduce RAM consumption of ODQA models (#1635).
• Removed code duplication and excessive parameters from classes derived from TorchModel (#1617).

Bug Fixes and Other Changes

• Fixed #1247, #1262, #1522 (#1655, #1656).
• Added wheel to requirements (#1650).

Breaking Changes

• Removed RabbitMQ connector (#1631).
• Removed overrides decorator (#1631).

Major Features and Improvements

• Improvement of models for question answering over knowledge graphs (#1598).
• GLUE models update (#1645).

Bug Fixes and Other Changes

• Fix the Russian SuperGLUE certificates bug by loading the datasets from DeepPavlov servers (#1633).
• General requirements update (#1631, #1643).
• Fixed CRF save in torch_transformers_sequence_tagger (#1637).
• Fixes in tests (#1640).

Major Features and Improvements

• Multitask learning implementation (#1627).

Breaking Changes

Removed Models

• FAQ models, tfidf_vectorizer, tfidf_logreg_autofaq_psearch and cv_tfidf_autofaq (#1608)

Removed Components

• ru_tokenizer (#1608)

Major Features and Improvements

• Python 3.10 support (#1614).
• Few-shot classification example and few-shot sampling support added to basic_classification_iterator (#1608).

Other Changes

• pymorphy2 is replaced with spacy in entity_linker, ru_adj_to_noun and answer_types_extractor components (#1618).

Fixes

• Fixed kbqa_cq_ru on Python 3.8/3.9 (#1609)

Documentation

• Added python inference pipelines example (#1613).

Other changes

• Updated model files upload script (#1607).

Major Features and Improvements

• Added -i/--install CLI argument and install argument to deeppavlov.build_model, deeppavlov.evaluate_model, deeppavlov.train_model to install model requirements before interaction with model (#1603).

Bug Fixes and Other Changes

• Reduced library verbosity: redundant logging info messages replaced with debug ones. Set nltk.download to quiet mode. (#1601).
• Replaced docs/features/models/classifiers.rst with docs/features/models/classification.ipynb. Fixed minor typos in documentation, removed skill concept (#1600).
• Removed /examples from the README.md links (#1602).

Breaking Changes

• Changed riseapi mode response format (#1585).
• Removed support for TensorFlow v1.x: removed all TF-based components, removed TF mentions from documentation, default train class replaced with torch_trainer (#1574).
• TensorFlow-based models were replaced with the PyTorch-based ones, some models were renamed, various models and components were removed.
  • Replaced Models:
    • Entity Linking (#1516)
    • Context Question Answering (#1539)
    • NER (#1545)
    • Classifier models (#1565)
  • Removed Models
    • Classifiers, Doc Retrieval, Go-Bot, Neural Morphological Tagging, NER, ODQA, Ranking, Spelling Correction, Context Question Answering (#1523)
    • ASR, TTS (#1526)
    • ELMO (#1533)
    • Chinese Context Question Answering (#1534)
    • Ranking Models (#1537)
    • Go-Bot (#1544)
    • KBQA, Multitask BERT (#1560)
    • Intent Catcher (#1564)
    • Morpho/Syntax Models (#1573)
  • Removed Components
    • Connectors to: Telegram, Microsoft Bot Framework, Amazon Alexa, Yandex Alice (#1548)
    • Various components (#1563)
• Removed serialization mechanism

Major Features and Improvements

• Python 3.8/3.9 support (#1525).
• Added nested configs overwriting mechanism (#1561).
• Added case-agnostic distil NER for DREAM (#1570).
• Added DeepPavlov Topics Classifier model (#1584).
• Added Russian SuperGLUE models (#1577).
• Added external metrics support (#1546).
• Added Jupyter Notebook support to documentation (#1592).

Bug Fixes and Other Changes

• Requirements updated (#1578).
• Models deprecation mechanism (#1547).
• Uploaded DeepPavlov BERT models with MLM & NSP heads parameters (#1502).
• Fixed en_core_web_sm loading error (#1524)
• Fixed NER models table view (#1529)
• Removed special version of Transformers library for certain components (#1532)
• Fixed tests (#1543)
• Updated library output during model training (#1572)
• Fixed ConnectionResetError handling in simple_download (#1586)
• Minor fixes in KBQA models (#1591)
• Added iterations count and speed output during training (#1593)
• Fixed datasets version (#1596)

• Changed links to group https://github.com/deeppavlov

• fixed broken links from https://github.com/deepmipt to https://github.com/deeppavlovteam

Note: DeepPavlov 1.0.0 is not released yet!

DeepPavlov 1.0.0 Release Notes

• Added Python 3.8 and 3.9 support and library requirements are optimized in #1525 and #1578.
• Removed all TensorFlow components and default trainer replaced with torch_trainer in #1574.
• Added Russian SuperGLUE models and submission generation in #1577.
• Added NER case-agnostig config in #1570.
• Added external metrics support in #1546.
• Nested config overwriting mechanism in #1561.
• Refactoring of the training logging in #1572.
• KBQA models migrated to PyTorch in #1569.
• Classification models migrated to PyTorch in #1565.
• NER models migrated to PyTorch in #1545.
• Context question answering models migrated to PyTorch in #1539.
• Entity Linking migrated to PyTorch and reduced RAM and VRAM consumption in #1516.
• Added config deprecation mechanism in #1547.
• torch_bert_ranker now uses the same Hugging Face Transformers version as the rest of the components in #1532.
• Models and components removed in #1523, #1526, #1534, #1533, #1537, #1544, #1560, #1563, #1564, #1573.
• Fixed a problem with pre-trained BERT models by DeepPavlov in #1502. Resolves #1275 and #1390.
• Fixed en_core_web_sm load error during tests in #1524.
• Removed Telegram, MSBot Framework, Yandex Alice and Amazon Alexa connectors in #1548.
• Documentation updated in #1517, #1529.

• fix TypeError: Descriptors cannot not be created directly. for Python 3.7.

• fix AttributeError module 'lib' has no attribute 'X509_V_FLAG_CB_ISSUER_CHECK.

Renamed models:

When a.json is renamed to b.json, original b.json is removed.

• squad_ru_torch_bert -> squad_ru_bert
• ner_rus_bert_torch -> ner_rus_bert
• insults_kaggle_bert_torch -> insults_kaggle_bert

TensorFlow replaced by PyTorch

• squad_ru_bert_infer

Removed models:

asr

• asr.json
• tts.json
• asr_tts.json

elmo

• rusentiment_elmo_twitter_cnn
• elmo_en_1billion
• elmo_ru_news
• elmo_ru_twitter
• elmo_ru_wiki

classifiers

• insults_kaggle
• insults_kaggle_conv_bert
• intents_dstc2
• intents_dstc2_bert
• intents_dstc2_big
• intents_sample_csv
• intents_sample_json
• intents_snips
• intents_snips_big
• intents_snips_sklearn
• intents_snips_tfidf_weighted
• relation_prediction_rus
• ru_obscenity_classifier
• rusentiment_bigru_superconv
• rusentiment_cnn
• sentiment_imdb_bert
• sentiment_imdb_conv_bert
• sentiment_sst_multi_bert
• sentiment_twitter_bert_emb
• sentiment_twitter_preproc
• sentiment_yelp_conv_bert
• sentiment_yelp_multi_bert
• sst_torch_swcnn
• topic_ag_news
• yahoo_convers_vs_info
• yahoo_convers_vs_info_bert

doc_retrieval

• en_ranker_tfidf_enwiki20161221

go_bot

• database_dstc2
• gobot_dstc2
• gobot_dstc2_best
• gobot_dstc2_best_json_nlg
• gobot_simple_dstc2

morpho_tagger

• morpho_ar
• morpho_cs
• morpho_de
• morpho_en
• morpho_es_ancora
• morpho_fr
• morpho_hi
• morpho_hu
• morpho_it
• morpho_ru_syntagrus
• morpho_ru_syntagrus_pymorphy
• morpho_ru_syntagrus_pymorphy_lemmatize
• morpho_tr

ner

• ner_conll2003_pos
• ner_dstc2
• ner_few_shot_ru
• ner_few_shot_ru_simulate
• ner_kb_rus
• ner_lcquad_bert_probas
• ner_ontonotes_m1
• slotfill_dstc2
• slotfill_dstc2_raw
• slotfill_simple_dstc2_raw
• slotfill_simple_rasa_raw
• vlsp2016_full

odqa

• en_odqa_infer_enwiki20161221

ranking

• paraphrase_ident_paraphraser
• paraphrase_ident_paraphraser_interact
• ranking_ubuntu_v2_mt
• ranking_ubuntu_v2_mt_interact

spelling_correction

• brillmoore_kartaslov_ru
• brillmoore_kartaslov_ru_custom_vocab
• brillmoore_kartaslov_ru_nolm

squad

• squad_bert_uncased
• squad_zh_bert_mult
• squad_zh_bert_zh

Removed components:

• ner_few_shot_iterator
• ru_obscenity_classifier
• snips_intents_iterator
• snips_ner_iterator
• snips_reader
• mpm_nn
• bilstm_gru_nn
• morpho_tagger
• siamese_predictor
• elmo_embedder
• ner_bio_converter
• ner_svm
• jieba_tokenizer
• base64_decode_bytesIO
• bytesIO_encode_base64
• nemo_asr
• nemo_tts

Other features:

• Python 3.8 and 3.9 support for non-tensorflow-based models
• Upload DeepPavlov BERT models with MLM & NSP heads parameters
• removed examples directory
• minor fixes and improvements

• Removed 12 configuration files and 5 components (1498, 1499)
• SuperGLUE models updated

• Pipeline building syntax using python is simplified

• A relation extraction model
• A ReCoRD model is based on Reading Comprehension with Commonsense Reasoning Dataset