Easily clean text with spaCy!
spacy-cleaner utilises spaCy
Language
models to replace, remove, and
mutate spaCy
tokens. Cleaning actions available are:
See our docs for more information
pip install -U spacy-cleaner
or install with Poetry
poetry add spacy-cleaner
spacy-cleaner can clean text written in any language spaCy
has a model
for:
import spacy
from spacy_cleaner import processing, Cleaner
model = spacy.load("en_core_web_sm")
Class Pipeline
allows for configurable cleaning of text using spaCy
. The
Pipeline
is initialised with a model and functions that transform spaCy
tokens:
cleaner = Cleaner(
model,
processing.remove_stopword_token,
processing.replace_punctuation_token,
processing.mutate_lemma_token,
)
Next the pipeline
can be called with the method clean
to clean a list of
texts:
texts = ["Hello, my name is Cellan! I love to swim!"]
cleaner.clean(texts)
The method clean
is a wrapper around the spaCy
Language
class method
pipe
. Check the docs for more information:
https://spacy.io/api/language#pipe
Giving the output:
['hello _IS_PUNCT_ Cellan _IS_PUNCT_ love swim _IS_PUNCT_']
You can see the list of available releases on the GitHub Releases page.
We follow Semantic Versions specification.
We use Release Drafter
. As pull requests are merged, a draft release is kept up-to-date listing the changes, ready to publish when youβre ready. With the categories option, you can categorize pull requests in release notes using labels.
Label | Title in Releases |
---|---|
enhancement , feature
|
π Features |
bug , refactoring , bugfix , fix
|
π§ Fixes & Refactoring |
build , ci , testing
|
π¦ Build System & CI/CD |
breaking |
π₯ Breaking Changes |
documentation |
π Documentation |
dependencies |
β¬οΈ Dependencies updates |
You can update it in release-drafter.yml
.
GitHub creates the bug
, enhancement
, and documentation
labels for you. Dependabot creates the dependencies
label. Create the remaining labels on the Issues tab of your GitHub repository, when you need them.
This project is licensed under the terms of the MIT
license. See LICENSE for more details.
@misc{spacy-cleaner,
author = {spacy-cleaner},
title = {Easily clean text with spaCy!},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/Ce11an/spacy-cleaner}}
}
This project was generated with python-package-template
This project was built using IntelliJ IDEA