colibri-core

[Ko van der Sloot]

• Major code cleanup: range-based for loops, override statement, convert most stream pointers to references, more const parameters

[Maarten van Gompel]

• cleanup: remove last traces of python2 support, refactored exceptions
• Set up wheel building

This release does not provide a shared library; use static linking instead.

• Python setup.py no longer attempts to build colibri core C++ library, must be done in a manual step before.
• Set up continuous integration and wheel building
• Python test fix

[Maarten van Gompel]

• Fixed long option parsing
• Fixed column length mismatch in TSV header/data output
• Fixed a build problem
• Updated installation instructions

[Ko van der Sloot]

• Significant cleanup of the code
• Updated for newer autoconfig versions

[Maarten van Gompel]

• codemeta.json: updating according to (proposed) CLARIAH requirements (CLARIAH/clariah-plus#38)
• Dockerfile: added

[Ko van der Sloot]

• Code cleanup
  • added some exceptions for unwanted cases detected by scan-build
  • out-dommented DOFLEXFROMCOOC and cached_DOFLEXFROMCOOC variables, they seem useless
  • removed unused assignments

Thanks to @kosloot, various warnings on clang were fixed in this minor release.

Implemented the ability to prune subsumed n-grams (retaining only the longer non-subsumed versions). Introduces a new PRUNESUBSUMED variable for PatternModelOptions.
Note: This is an aggressive form of pruning that should also work for unordered models, matching is based on types rather than individual tokens (all subsumed types are pruned).

Bugfix release: Certain options from PatternModelOptions were not available to the python binding yet.

Bugfix release: Pattern size and category constraints were not working for several methods (getcooc/getleftcooc/getrightcooc/getleftneighbours/getrightneighbours) #44

Very minor update release:

• Updated codemeta metadata
• Added ClassEncoder.find()

Better handling of large patterns, PatternPointer size descriptor is now 64 bits (fixes #42) at cost of a small increase in memory consumption in various computations.

(The experimental and relatively unused PatternPointerModels are not backwards compatible, contact me if this is a problem)

Important bugfix release:

• Fixes data-clipping bug on loading large corpora in memory (used by indexed patternmodels) #41

(All users are urged to upgrade!)

• Added metadata
• macOS fix

• Minor update: made setup.py more robust for manual installation mode (without compiling C++ lib) (v2.4.7 was skipped)

• fix: colibri-classencode -t (threshold) behaviour was wrong (was interpreted as +1)

• Refactored alignment model
• added BasicPatternAlignmentModel
• Major cleanup of warnings and possible issues (thanks to @kosloot)

• Bugfix: fixes covered token count per category/n (issue #26)
• New feature: colibri-patternmodeller has a--simplereport (-r) option that generates a report without coverage information (more limited but a lot faster)

v2.4.2 was prematurely released, one minor test was corrupt. Fixed now in this release.

Bugfix release, fixes issue #25

Minor fix release prior to paper publication:

• Python 2.7 compatibility fix
• Updated python tutorial
• Added benchmarks

Various fixes:

• Speed up in ngrams() computation (issue #21)
• Performance fix for processing long lines
• Pattern.instanceof()should be faster and is now available from Python too
• Attempt to fix compilation issue on certain platforms (issue #22), unconfirmed

New features:

• Implemented new filtering mechanism that supports actively checking whether patterns are instances of a limited set of specified skipgrams, or a superset of specified ngrams.
• Implemented ignorenewlines option in class encoding. Useful if you have source text split by for instance sentences (one per line), but want a model that crosses sentence boundaries.
• Implemented vocabulary import for the class encoding stage (issue #2)