Changelog

New processors

• Added a selective extraction processor that allows to write field values of a given log message to a different Kafka topic.
• Added a list comparison enricher processor that allows to compare field values against lists as files.
• Added a domain label extractor processor that allows to split a given domain into it's parts/labels.

Changes to processors

• Multiple ip-alerter files can now be used in the predetector.
• Added dotted output fields for the geoip enricher processor. This allows to save the geoip data in custom subfields, depending on the configured dotted output field.
• Implemented a unix source format feature for the timestamp normalization. With this it is possible to use 'UNIX' in a normalization rule to normalize a unix epoch timestamp. This works with seconds and milliseconds.
• Modified the geoip_enricher and domain_resolver to allow for configurable output fields (per rule basis). Both still use the default output fields as default values in case none is provided.
• Optimized the clusterer by compilng regex patterns and preventing an endless loop.
• Fixed problem in datetime extractor tests caused by daylight savings.

Changes to connectors

• The kafka connector can now internally update offsets after a record has been processed instead of updating it for the whole batch, but it does still update it periodically on kafka for performance reasons.
• Added a hmac to an incoming event from kafka. This modification to the confluent kafka connector allows to verify the integrity of an incoming message.

Other changes

• The version in setup.py is now determined via versioneer by using git tags.
• Added calculation of rule test coverage to the auto rule tester. This allows to get an overview of how many rules have unit tests.
• Added a CI pipeline via GitHub actions.
• Added a push mirror via GitHub actions.
• Updated the pinned the requirements.
• Added more docstrings.
• Refactored the code.