spacy-extreme

An example of how to use spaCy for extremely large files without running into memory issues

Stars
35
View Code on GitHub View on X
Ecosystems: Python

Statistics for this project are still being loaded, please check back later.

Related Projects
embedding-reader

Efficiently read embedding in streaming from any filesystem

27 Feb 2022 94
GPT2

An implementation of training for GPT2, supports TPUs

13 May 2019 1,419
minimal-llama

06 Mar 2023 442
SmartLMVocabs

Improving Language Model Performance through Smart Vocabularies

22 Nov 2018 6
spaCy

๐Ÿ’ซ Industrial-strength Natural Language Processing (NLP) in Python

03 Jul 2014 29,548
starcoder2

Home of StarCoder2!

08 Dec 2023 1,732
neuralcoref

โœจFast Coreference Resolution in spaCy with Neural Networks

03 Jul 2017 2,849
spacy_conll

Pipeline component for spaCy (and other spaCy-wrapped parsers such as spacy-stanza and spacy-udpi...

19 Dec 2018 72
starcoder

Home of StarCoder: fine-tuning & inference!

24 Apr 2023 7,267
Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

02 Jul 2021 1,323
korean-spacing-model

ํ•œ๊ตญ์–ด ๋ฌธ์žฅ ๋„์–ด์“ฐ๊ธฐ(์‚ญ์ œ/์ถ”๊ฐ€) ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค. ๋ฐ์ดํ„ฐ ์ค€๋น„ ํ›„ ์ง์ ‘ ํ•™์Šต์ด ๊ฐ€๋Šฅํ•˜๋„๋ก ์ž‘์„ฑํ•˜์˜€์Šต๋‹ˆ๋‹ค.

16 Sep 2020 54
languagemodels

Explore large language models in 512MB of RAM

07 May 2023 1,154
bilm-tf

Tensorflow implementation of contextualized word representations from bi-directional language models

29 Sep 2017 1,620
LongMamba

Some preliminary explorations of Mamba's context scaling.

03 Feb 2024 177
two_hot_encoding

17 Jan 2022 6