pySigma backend for generating Grafana Loki/LogQL rules
OTHER License
This is the Loki backend for pySigma. It provides the package sigma.backends.loki
with the LogQLBackend
class.
It supports the following output formats for Sigma rules:
default
: plain Loki LogQL queriesruler
: creates Loki LogQL queries in the ruler (YAML) format for generating alertsIt also supports the following query formats for and categories of Sigma Correlation rules:
default
format using LogQL metric queries:
event_count
value_count
It includes the following pipeline transformations in sigma.pipelines.loki
:
SetCustomAttributeTransformation
: adds a specified custom attribute to a rule, which can be used to introduce a stream selector or parser expression into the generated query
LokiCustomAttributes
enum contains the relevant custom attribute names used by the backendFurther, it contains the processing pipelines in sigma.pipelines.loki
:
loki_log_parser
: converts field names to logfmt labels used by Grafanaloki_promtail_sysmon
: parse and adjust field names for Windows sysmon data produced by promtail
sysmon
service tag, and hence this pipeline should be used in combination with the generic sysmon pipeline
loki_okta_system_log
: parse the Okta System Log event json, adjusting field-names appropriatelyWhen converting rules into queries, the backend has the following optional arguments:
add_line_filters
(boolean, default: False
): if True
, attempts to infer and add new line filters to queries without line filters, to improve Loki query performance
case_sensitive
(boolean, default: False
): if True
, defaults to generating case-sensitive query filters, instead of case-insensitive filters that the Sigma specification expects, trading between Loki query performance and potentially missing data with unexpected casing
False
, as these versions of Loki may contain issues with case-insensitive filters, which cause such queries to fail to match desired dataThis backend is currently maintained by:
To get started developing/testing pySigma-backend-loki, these steps may help you get started:
poetry install
to install the Python dependenciespoetry shell
to activate the poetry environmentpoetry run pytest
git config --local core.hooksPath .githooks/
To release new versions of pySigma-backend-loki, we use GitHub actions to update PyPI. When the main branch is in state that is ready to release, the process is as follows:
\d+\.\d+\.\d+(-[0-9A-Za-z-]+)?
git tag --sign --message="Release vX.X.X" vX.X.X
git push --tags
, and validate that the release to the test instance of PyPI is successfulpoetry build
to produce distributable versions in dist/
v0
, or ends with -alpha/beta
etc., mark it as a pre-release, and attach the distributable files to the releasepySigma
, do a pull request on the pySigma-plugin-directory to reflect thatThese features are currently either WIP or are planned to be implemented in the near future.
These features are not easily supported by the backend, and hence are unlikely to be implemented.