foliatools

• Fixed excessively slow foliaupgrade #7

Significant fixes and improvements for tei2folia (was previously only tested on DBNL collection, now on several others too).

• Added txt2folia tool
• [conllu2folia] added --outputfile parameter

• [alpino2folia] Fixed conversion of dependencies and syntax layer, was missing or a long time due to an unforeseen change in Alpino output (proycon/folia#49)
• [foliaspec] Added code generation for new Rust library
• [foliabench] Added a new benchmark tool for the foliapy library, implementing four benchmarks.

Documentation update, added docstrings to all the tools for documentation (automatically harvestable for metadata)

Bugfix release:

• [tei2folia] prevent generating empty folia comments in rare cases (La…
• [tei2folia] fix in t-ref, text ended up outside attribute
• [folialangid] language constraints were not propagated to underlying …
• [folialangid] implemented confidence threshold and fallback language

Minor bugfix release:

• [folialangid] exit code was wrong

• [folialangid] adding support for using a different textclass as source
• [folialangid] fix in conflicting options for folialangid
• [tei2folia] Small adaptation for part annotation and spacing (proycon/folia#61)

• [foliamerge] fixing breakage with new library (reported in proycon/flat#148)
• [foliaupgrade] exit code was wrong

Previous release was broken, fixed now.

• added language detection tool (folialangid) based on lang.py
• --fixunassignedprocessor didn't work properly yet

• [foliaupgrade] Added --fixunassignedprocessor flag to fix a known issue with some FoLiA documents

• [tei2folia] prevent creating a text content element with only one (or more) non-breaking spaces (was not handled by normalize-space() as expected)
• [foliaupgrade] Migrated part of the folia upgrader to the foliapy library itself

Minor fixes for conllu2folia and tei2folia

• Implemented a tei2folia tool, implementing a subset of TEI (tested on the DBNL collection by the KB). Loosely based on prior work by @JesseDeDoes.

• added a new foliaerase tool that can remove certain annotation types
• foliaupgrade: various fixes
• rst2folia: adding a wrapper for FLAT integration
• conllu2folia: adding a wrapper for FLAT integration

• foliaupgrade: refactoring and updates
• foliaupgrade: set src and format attributes on pre-upgraded version
• foliavalidator: add proof of validation (not cryptographical) to provenance chain (only when --output is chosen)
• folia2html: updated for FoLiA v2

• foliacat: More provenance data for foliacat
• foliacat: Compatibility for serialising older FoLiA versions
• Added conllu2folia tool (#1)
• implemented foliaupgrade, convert old-style annotator data to provenance

Minor refactoring of foliacat and better version reporting

• Fix for foliacat when dealing with older versions