3.14.1

Features

• Password protected ssl keys (#1888)
• Add OpenBSD module to system() source (#1875)
• Add Ubuntu Trusty support to Docker build (#1849)

Bugfixes

• Fix increased memory usage during saving disk-buffer (#1867)
• Fix maximum record length limitations of disk-buffer (#1874)
• Fix a memory leak in cfg-lexer (#1843)
• Fix some issues found by pylint in python module (#1881, #1830)
• Fix a crash due to a race condition in kv-parser() (#1871)
• Fix a crash due to a race condition in file() destination (#1858)
• Fix deprecated API usage in python module tests (#1829)
• Fix a race condition in internal() source (#1815)
• Fix a locale issue in merge-grammar python tool (#1868)
• Fix compile problems with autotools when '--disable-all-modules' used (#1853)
• Fix a file descriptor leak in persist-state (#1847)
• Fix a file descriptor leak in pseudofile() (#1846)
• Fix memory/fd leaks in loggen tool (#1844, #1845)
• Fix compile problems on Fedora, RHEL6, CentOS6 and SUSE based platforms (#1837)
• Fix a crash when large variety of keys added to messages (#1836)
• Fix compile problems when PATH_MAX not defined (#1828)
• Fix integer overflow problems in grammar (#1823)
• Fix a memory leak in filter() (#1812)
• Fix memory leak of persist-name() option (#1816)
• Fix message corruption caused by a bug in the subst() rewrite rule (#1801)
• Fix silently dropped messages in elasticsearch2() when sending in bulk mode (#1800)
• Fix broken disk-buffer() support in elasticsearch2() (#1807)
• Fix Hy support in python module (#1754)
• Fix an event scheduler related crash during reloading syslog-ng (#1711)
• Fix a crash with SIGBUS when persist file cannot grow (#1785)

Other changes

• Improve error reporting in "block" definitions in config (#1809)
• Add warning message when disk-buffer() directory is changed in configuration (#1861)
• Syslog-ng debun improvements (#1840)
• Refactor in rewrite() module init (#1818)
• Missing child program (exit status 127) handling is changed in program() destination:
  stopping destination instead of polling for the child program (#1817)
• Refactor in filter() module (#1814)
• Improve thread synchronization in mainloop and refactor (#1813)
• Adapted json-c v0.13 API changes to json-parser (#1810)
• Add filters as selectors in contextual data (#1838)

Notes to the developers

• Full cmake support achieved (#1777, #1819, #1811, #1808, #1805, #1802, #1841, #1806)
• Add support for modules to have module specific global options (#1885)
• Improved MacOS support (#1862, #1864, #1865)
• Add new option to exclude directories in style-checker tool (#1834)
• Ivykis dependency updated to 0.42.2 release (#1711)
• Journald grammar, source and header files are part of dist tarball (#1852)
• Add valgrind support for unit tests (#1839)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Antal Nemes, Balazs Scheidler, Björn Esser, Fabien Wernli, Gabor Nagy, Gergely Nagy,
Janos Szigetvari, Juhász Viktor, Laszlo Budai, Laszlo Szemere, László Várady, Orion Poplawski,
Attila Szalay, Shen-Ta Hsieh, Tamas Nagy, Peter Kokai, Norbert Takacs, Zoltan Pallagi.

3.13.2

Fixes

• Missing manpages from release tarball (#1793)
• Package syslog-ng-mod-json is removed from (#1794)
• Drop syslog-ng-abi virtual packages (#1797)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Gergely Nagy, Laszlo Budai, Laszlo Varady, Peter Czanik.

3.13.1

Features

• Add app-parser() framework (automatic parsing of log messages) (#1689)
• Support microseconds in Riemann destination (#1710)
• Add osquery destination as an SCL plugin (#1728)
• Add network load balancer destination (#1706)
• Add possibility to only signal re-open of file handles (SIGUSR1) (#1530)
• It is possible from now to limit the number of registered dynamic counters (#1743)
• Add $(binary) template function (#1679)
• Add experimental transport for transferring messages in whole between syslog-ng instances (EWMM) (#1689)
• Docker based build and debian package generation (#1783)
• Add auto-parse(yes/no) to app-paser(), system() and default-network-drivers() (#1788)
• Add Graylog2 destination and $(format-gelf) template function (#1680)

Bugfixes

• Exit when a read fails on an included config file instead of
  starting up with an empty configuration. (#1721)
• Fix double free (#1720)
• Add missing discarded counter to groupingby (#1748)
• Fix a reference leak in Python destination (#1716)
• Fix timezone issue in snmptrapd parser (#1746)
• Fix potential crash in stdin driver (#1741)
• Fix a crash when initializing new config fails for socket with keep_alive off (#1723)
• Fix filter evaluation in case of contexts with multiple elements (#1718)
• Various grouping-by fixes (#1718)
• Fix potential use after free around dns-cache during shutdown (#1666)
• Fix access to indirect values within Java destination (#1732)
• Fix a crash in affile (#1725)
• Fix a memory leak (#1724)
• Fix a crash when getent is used empty group (#1691)
• Fix jvm-options() (#1704)
• Fix a crash in Python language binding (#1694)
• Fix a crash in afmongodb (#1765)
• Fix a memory leak in afmongodb (#1766)
• Fix name-to-GID calculation in the $(getent) template function (#1764)
• Fix a crash when redis is configured without the command() option (#1767)
• Fix a race condition in kv-parser() (#1789)

Other changes

• Cleanup diskq related warning messages (#1752)
• Provide tls block for tls options in amqp(), http(), riemann() destination drivers (#1715)
• It it possible from now to register blocks and generators as plugins (#1657)
• Drop compatiblity with configurations below 3.0 (#1709)
• Do not change permissions of a file by default (#1782)
• Allow source files to specify permissions locally (#1782)
• Minor performance improvement (#1729)
• The current config version can be queried with "--version" (#1740)
• Increase the performance of kv-parser() (#1789)

Notes to the developers

• Change configure default option for jsonc and mongoc from auto to internal (#1735)
• Disable ASLR when running unit tests (#1753)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Antal Nemes, Attila Szalay, Balazs Scheidler, Gabor Nagy,
Jakub Jankowski, Janos Szigetvari, Laszlo Budai, Laszlo Varady, Laszlo Szemere,
Marton Illes, Mate Farkas, Peter Kokai, Pontus Andersson, Sam Stephenson,
Sebastian Roland, Viktor Juhasz, Zoltan Pallagi.

3.12.1

Features

• HDFS: support macro in filename (#1638)
• HDFS: add append support (#1675)
• Java: allow to use sequence numbers in templates (#1628)
• TLS improvements (#1603, #1636)
  • Add PKCS 12 support with the new pkcs12-file() TLS option
  • startup time ssl-options() and peer-verify() check
  • startup time key_file, cert_file, ca_dir, crl_dir and cipher_suite check
  • ECDH cipher support (OpenSSL 1.0.1, 1.0.2, 1.1.0) with the ecdh-curve-list() option (only available >= 1.0.2)
    • for < 1.0.2, a hard-coded curve is used
    • for >= 1.0.2, automatic curve selection is used (the ecdh-curve-list() option can restrict this list)
  • DH cipher support with the dhparam-file() option
    • if the option is not specified, fallback RFC 3526 parameters are used
  • minor fixes
• stdin() source driver (#1605)
• Implement read_old_records option for systemd-journal source (#1642)
• Add tags-parser: a new module to parse $TAGS values (#1658)
• Add a Windows eventlog parser scl module (#1572)
• Add XML parser module (#1659, #1684)

Bugfixes

• Fix cannot parse ipv6 into hostname (#1617)
• Speedup add-contextual-data by making ordering optional (#1645)
• Fix monitor-method() option not working for wildcard-file() source (#1651)
• Sanitize SDATA keys in syslog-protocol messages to avoid generating non-valid messages (#1650, #1654)
• Fix memory leaks reported using Valgrind (#1649)
• Fix memory leak related to cloning pipes and reload (#1647)
• Fix getent protocol number returns incorrect value (#1665)
• Fix elasticsearch2 destination flush mechanism (#1668)
• Fix file destination related memory leak (#1685)
• Fix a possible memory leak around affile destination (#1685)

Other changes

• Improve syslog-ng debun functionality (#1633, #1641, #1663)
• Java: allow to set JVM options form global syslog-ng options (#1639)
• Do steps towards Python 3 support:
  • Fix string compatibility for Python 3 (#1632)
  • Improve Python version auto detection (#1660)
• HTTP destination: display verbose logs on debug level (#1526)
• Improvements for Solaris packing (#1664)

Notes to the Developers

• Update internal RabbitMQ (#1662)
• Update internal ivykis to v0.42 (#1566)
• Fix Travis and test related issues (#1566, #1644, #1674)
• Update docker images (#1637)
• Fix some clang compile errors (#1662)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:
Andras Mitzki, Antal Nemes, Attila Szalay, Balazs Scheidler, Gabor Nagy,
Gergely Orosz, Janos Szigetvari, Laszlo Budai, Laszlo Varady, Mate Farkas,
Marton Suranyi, Peter Kokai, Szilard Pfeiffer, Tamas Nagy, Zoltan Pallagi.

3.11.1

Features

• Add geoip2 parser and template function.
  It is based on the libmaxminddb(MaxMindDB).
  It will replace the old geoip parser and template function,
  so they are deprecated from 3.11 (but still available).

• Add SSL support to AMQP.

• Add template option to apache-accesslog-parser.

• Add configurable event time to Riemann destination.

• Add drop-unmatched() option to dbparser.

• Add Ubuntu Xenial to the bundled docker images.

• Support multi-instance support for Solaris 10 and 11.

• Support multi-instance for systemd.

• Add configurable timeout to HTTP destination.

• Add prefix() option to cisco-parser.

Bugfixes

• Fix a memory usage counter underflow for threaded destination drivers
  and writers.

• Fix a potential crash in AMQP.

• Fix a potential crash during reload.

• Fix a reload/shutdown issue.
  Under heavy load, worker might never exit from the fetch loop from the
  queue.

• Fix a potential crash in afsocket destination during reload.

• Fix a counter registration bug.
  In some cases not all the required counters are registered.

• Fix a build issue on FreeBSD.

• Fix a memory leak in diskq plugin.

• Fix systemd-journal error codes validation.

• Fix a potential crash in diskq when it is used with file
  destination and the file is reaped.

• Fix a memory leak in HTTP destination

• Fix ENABLE_DEBUG in dbparser.

• Fix a unit tests that caused build issue on 32 bit platforms.

Other changes

• The eventlog library is part of syslog-ng from now.

• Improve error messages when the config cannot be initialized.

• Improve source suspended/resumed debug messages.

• Rename syslog-debun to syslog-ng-debun.

• Update manpages to v3.11

• Remove tgz2build directory.

Notes to the Developers

• Rewrite merge-grammar script from Perl to Python.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Antal Nemes, Attila Szalay, Balazs Scheidler, Fabien Wernli,
Gabor Nagy, Giuseppe D'Anna, Janos Szigetvari, Laszlo Budai, Laszlo Varady,
Lorand Muzamel, Mate Farkas, Noemi Vanyi, Peter Czanik, Tamas Nagy,
Tibor Bodnar, Tomasz Kazimierczak, Zoltan Pallagi

3.10.1

Features

• Support https in http (curl) module

• Docker support : from now Dockerfile for CentOS7, Ubuntu Zesty and for
  Debian Jessie is part of our upstream

• Add --database parameter for geoip template function

• Metric improvements

  • add discarded messages for parsers
  • add matched/not matched counter for filters
  • add memory_usage counter to logqueue
  • add written counter
    • Written is a calculated counter which return the written messages
      by destinations. Written message is which was processed but not
      queued and not dropped. (written = processed - queued - dropped)
  • stats-counters: rename stored counter to queued
  • add global_allocated_logmsg_size counter for tracking memory logmsg
    related allocations

• Add snmp-parser (v1, v2)

  • parses snmptrapd log
  • The parsed information is available as key-value pairs, which can be
    used/serialized (macros, format-json, etc.) in the log path.
    If you want to send the message in a structured way, you can disable the
    default message generation with the generate-message(no) option.

• Add snmp-soure

  • available as an SCL block that containing a filesource and an SNMP parser
    modules: add snmptrapd parser

• Add osquery source

  • available as an SCL block
  • It reads the osquery log file and parses with the JSON parser,
    creating name-vaule pairs with an .osquery. prefix by default.

• Add cisco-parser

  • available as an SCL block

• Add wildcard filesource

• Add startdate template function

• Add $(basename) and $(dirname) template functions

• Add Kerberos support for HDFS destination

• Add AUTH support for redis destination

• Add map-value-pairs() parser

  • it can be used to map existing name-value pairs to a different set during
    processing, in bulk. Normal value-pairs expressions can be used, just
    like with value-pairs based destinations.

• Extend Python language binding by Python parser

• Add support for extract-stray-words() option in kv-parser()

  • stray words: those words that happen to be between key-value pairs and
    are otherwise not recognized either as keys nor as values.

• Add $(context-values) template function

• Add $(context-lookup) function

• Add list related template functions

  • $(list-head list ...) returns the first element (unquoted)
  • $(list-nth NDX list ...) returns the specific element (unquoted)
  • $(list-concat list1 list2 ...) returns a list containing the concatenated
    list
  • $(list-append list elem1 elem2 ...) returns a list, appending elem1,
    elem2 ...
  • $(list-tail list ...) returns a list containing everything except
    for the first element
  • $(list-slice FROM:TO list ...) returns a list containing the slice
    [FROM:TO), Python style slice
    boundaries are supported (e.g. negative)
  • $(list-count ...) returns the number of elements in list

• Add add query commands to syslog-ng-ctl

  • query list List names of counters which match the filter
  • query get Get names and values of counters which match the
    filter
  • query get --sum Get the sum of values of counters which match the
    filter

• Support multiple servers in elasticsearch2-http destination

• Implements elastic-v2 https in http mode

• Add getent module (ported from incubator)

  • This module adds $(getent) that allows one to look up various NSS based
    databases, such as passwd, services or protocols.

• Add support for IP_FREEBIND

Bugfixes

• Fix a libnet detection check error that caused problem configuring
  enable-spoof-source.

• Avoid warnings about _DEFAULT_SOURCE on recent glibc versions
  With the glibc on zesty, using _GNU_SOURCE and not defining _DEFAULT_SOURCE
  results in a warning, avoid that by defining _DEFAULT_SOURCE as well.

• Fix invalid database warning for geoip parser

• Fix prefix() default in systemd-journal for new config versions

• Fix a potential message loss in Riemann destination

• Fix a potential crash in the Riemann destination when the client is not
  connected to the Riemann server.

• Fix a possible add-contextual-data() related data loss in case of multiple
  reference to the same add-contextual-data parser in several logpaths.

• Fix dbparser deadlock

• Fix Python destination

  • open() was not called in every time_reopen()
  • python destination is not defined in stats output

• Fix processed stats counter for afsocket

• Fix stats source for pipes

  • Previously pipe source is shown as file

• Fix csv-parser multithreaded support
  In some cases (when csv-parser attached to network source), the parser
  randomly filled the column macros with garbage.

• Fix a message loss in case of filesource when syslog-ng was restarted and
  the log_msg_size > file size.

• Fix a potential crash in cryptofuncs

• Fix a potential crash in syslog-ng-ctl when no command line parameters was
  set.

• Fix token duplication in the output of '--preprocess-into'

• Fix UTF-8 support in syslog-ng-ctl

• Fix a potential crash during X.509 certificate validation.

• Fix a segfault in Python module startup

• Fix a possible endless reading loop issue in case of multi-line filesource.

• Fix soname for the http module from "curl" to "http"

• Avoid openssl 1.1.0 deprecated APIs
  When openssl is built with --api=1.1 disable-deprecated, use of deprecated
  APIs results in build failure.

Other changes

• Increase processed counter by queued counter after reload or restart when
  diskqueue is used otherwise the newly added written counter would underflow.

• Set the default time-zone to UTC for elasticsearch2
  Elasticsearch and Kibana use UTC internally.

• Add retries support for python destination

• Prefer server side cipher suite order

• Always include librabbitmq in the dist tarball

• Always include ivykis in the dist tarball

• Marking parse error locations with >@<.

• Default log_msg_size is increased to 64Kbyte from 8Kb

• Tons of syslog-debun improvements

• Exit with 0 return code when --help is specified for syslog-ng-ctl

• syslog-ng: make '--preprocess-into' foreground only

• Add debug messages on log_msg_set_value()

• Add more detail to filter evaluation related debug messages

Notes to the Developers

• Extract template perf test function to testlib

• Print a debug message when logmsg passed to the Python side

• Allow http module (curl) to be build with cmake

• astylerc: allow continuation lines to start until column 60

• Move kv-scanner under syslog-ng/lib

• scratch-buffers2: implement an alternative to current scratch buffers
  This new API is aimed a bit easier to use in situations where a throw away
  buffer is needed that will automatically be freed at the next message.
  It also gets does away with GTrashStack that is deprecated in recent glib
  versions.

• Several refactors in stats module.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Antal Nemes, Balazs Scheidler, eroen, Fabien Wernli, Gabor Nagy,
Gergely Nagy, Janos Szigetvari, Jason Hensley, Laszlo Varady, Laszlo Budai, Mate Farkas,
Noemi Vanyi, Peter Czanik, Peter Gervai, Todd C. Miller, Philip Prindeville,
Zoltan Pallagi

3.9.1

Features

• Improve parsing performance in case of keep-timestamp(no), earlier the
  timestamp was parsed and then dropped, now we don't parse it, which is a
  2x performance improvement in reception speed.

• TLS based transports will publish the peer's certificate in a set of
  name-value pairs, as follows:

  • .tls.x509_cn - X.509 common name
  • .tls.x509_o - X.509 organization string
  • .tls.x509_ou - X.509 organizational unit

• Improve performance of the tcp() source, due to a bug, syslog-ng
  attempted to apply position tracking to messages coming over a TCP
  transport, which is used for file position tracking and causing
  performance degradation. This bug is fixed, causing performance to be
  increased. (#1195)

• Make it possible to configure the listen-backlog() for any stream based
  transports (unix-stream and tcp). Earlier this was hard-wired at 256
  connections, now can be tuned using an option. For example:

  tcp(port(6514) listen-backlog(2048));

• Add a groupunset() rewrite rule that pairs up with groupset() but instead
  of setting values it unsets them. (#1235)

• Add support for Elastic Shield (#1228) and SearchGuard (#1223)

• kv-parser() is now able to cope with unquoted values with an embedded
  space in them, it also trims whitespace from keys/values and is in
  general more reliable in extracting key-value pairs from arbitrary log
  messages.

• Improve performance for java based destinations. (#1243)

• Add prefix() option to add-contextual-data()

Bugfixes

• Fix a potential crash in the file destination, in case it is a template
  based filename and time-reap() is elapsed. (#1183)

• Fix a potential ACK problem within syslog-ng that can cause input windows
  to overflow queue sizes over time, effectively causing message drops that
  shouldn't occur. (#1230)

• Fix a heap corruption bug in the DNS cache, in case the maximum number of
  DNS cache entries is reached. (#1218)

• Fix timestamp for suppression messages. (#1233)

• Fix add-contextual-data() to support CRLF line endings in its CSV input
  files.

• Fixed key() option parsing in riemann() destinations.

• Find libsystemd-journal related functions in both libsystemd-journal.so
  and libsystemd.so, as recent systemd versions bundled all systemd
  related libs into the same library.

• Fixed the build-time detection of system-wide installed librabbitmq,
  libmongoc and libcap.

• Fix the file source to repeatedly check for unexisting files, as a bug
  caused syslog-ng to stop after two attempts previously. (#841)

• The performance testing tool "loggen" crashed if it was used to generate
  messages on multiple threads over TLS. This was now fixed. (#1182)

• Fix an issue in the syslog-parser() parser, so that timestamps parsed
  earlier in the log path are properly overwritten. Earlier a time-zone
  setting may have remained in the timestamp in case the first timestamp
  did contain a timezone and then the one parsed by syslog-parser() didn't.
  (#1206)

• Due to a compilation issue, tcp-keepalive-time(), tcp-keepalive-intvl() and
  tcp-keepalive-probes() were not working, now they are again. (#1214)

• The --disable-shm-counters option is now passed to mongo-c-driver to work
  around a minor security issue (#1219).

  https://jira.mongodb.org/plugins/servlet/mobile#issue/CDRIVER-1691/comment/1405406

• Fix compilation issues on FreeBSD. (#1252)

• Add support to month names in all caps in syslog timestamps. At least one
  device seems to generate these. (#1263)

• The options() option to java destination can now accept numbers and not
  just strings.

• Fix a memory leak in the java destination driver, that may affect java
  based destinations like ElasticSearch, Kafka & HDFS.

Other changes

• HDFS was updated to 2.7.3
• Elasticsearch was updated to 2.4.0
• Support was added for OpenSSL 1.1.x (#1281)

Notes to the Developers

• We started to standardize our tests on the criterion unit testing
  framework, please submit all new tests using this framework. Patches to
  convert existing ones are also welcome.
  https://github.com/Snaipe/Criterion
• We also added a configuration file for astyle and accompanying make
  targets to check/reformat the source code to meet syslog-ng's style.
• debian/ directory has been removed from the "master" branch and is now
  maintained in a separate "release" branch.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Lászlo Várady, 0xddaa, Balázs Scheidler, Tamás Nagy, László Budai,
Fabien Wernli, Viktor Juhász, Kyle Manna, Michael Wimpy, Noémi Ványi,
Attila Szalay, Tibor Bodnár, Zoltán Pallagi

3.8.1

Library updates

• Kafka-client updated to version to 0.9.0.0
• Minimal required version of hiredis is set to 0.11.0 to avoid
  possible deadlocks
• Minimal version of libdbi is set to 0.9.0

New dependencies

• From now autoconf-archive package is a build-dependency.

Improvements and features

• Added the long-waited disk-buffer.
• date-parser ported from incubator to upstream
• New template functions: min, max, sum, average
• Added Apache-accesslog-parser
• Added loggly destination
• Added logmatic destination
• Added template function for supporting CEF.
• cURL-based HTTP destination driver added (implemented in C
  programming language)
• SELinux policy installer script now has support for Red Hat Enterprise Linux/CentOS/
  Oracle Linux 5, 6 and 7.
• Implemented add-contextual-data:
  With add-context-data syslog-ng can use an external database file to append
  custom name-value pairs on incoming logs (to enrich messages). The 'database'
  is actually a file that containing <selector, name, value> records.
  Currently only CSV format is supported.
  It is like geoip parser where the selector is $HOST, but in this case,
  the user can define the selector, and also the database contents.

Drivers

• Program destination/source drivers
  • Added inherit-environment configuration option to program source and
    destination. When it is set to true then the process will inherit the
    entire environment of the parent process.
  • Added keep-alive option to program destination (afprog).
    This option will control whether the destination program should be
    terminated at reload, or should be left running.
• Java drivers
  • HTTP destination
    • Added the ability to use templates in both url and message.
  • ElasticSearch Destination driver :
    • Support 2.2.x series of ElasticSearch (transport and node mode) .
    • Support Shield plugin for both ElasticSearch 1.x and ElasticSearch
      2.x .
    • Implemented new mode (HTTP) that can work with ElasticSearch 1.x,
      ElasticSearch 2.x, and even with Elastic 5. HTTP mode is based on
      a Java HTTP Rest client (Jest : https://github.com/searchbox-io/Jest).
      Note: make install will copy Jest library to the syslog-ng install
      directory.
• MongoDB destination driver
  • Replaced submodule limongo-client with mongo-c-driver.
  • Additional support for previous syntax used by libmongo-client
    before we started using mongo-c-driver and its URI syntax exclusively.
    Note that these are plainly translated to a connection URI without
    much sanity checking or preserving their former semantic meaning.
    So various aspects of the MongoDB connection like health checks, retries,
    error reporting and synchronicity will still follow the slightly altered
    semantics of mongo-c-driver.
• Riemann destination driver
  • Use cert-file() and key-file() options to match afsocket keywords as
    the same way as afsocket drivers use these options. The old one still
    work though.

Rewrite rules

• Introduced template options in rewrite rules.
• Added unset operation to make it possible to unset a specific name-value pair
  for a logmessage.

Parsers

• kvformat: make it possible to specify name-value separator
• linux-audit-scanner: recognize a0-a9* as fields to be decoded
  Argument lists are encoded in a0, a1, ... fields that can potentially be
  hex-encoded.
• csv-parser has been refactored, extended with new dialect and prefix options.
  Dialect is to convey CSV format information, instead of using flags
  Prefix option gets prefixed to all column names, just like with
  other parsers.

PatternDB

• added groupingby() parser that can perform simple correlation on
  log messages. In a way it is similar to the SQL GROUP BY operation, where
  an aggregate of a set of input records can be calculated.
  The major difference between SQL GROUP BY and groupingby() is that the
  first always operates on a enumerable list of records, whereas
  groupingby() works on a stream of data.
  A few use-cases where this can be useful:
  • Linux audit logs
  • postfix logs
• added create-context action
  Added a new possible action in the element, to create
  a new correlation context out of the current message and its associated
  context. This can be used to "split" a state.
• Added NLSTRING parser that captures a string until the
  following end-of-line. It can be used in patterns as: @NLSTRING:value@
  It doesn't expect any additional parameters. This makes it pretty easy to
  parse multi-line Windows logs.

Miscellaneous features

• syslog-debun (debug bundle script for syslog-ng) has been improved

Bugfixes

• geoip-parser: When default database if not specified, syslog-ng crashed.
• Added support for multiple drivers with the same name in syslog-ng config.
• Fixed aack counting logic for junctions that have branches that modify
  the LogMessage.
• Fixed a potential crash for code that uses log_msg_clear()
  in production (e.g. syslog-parser()).
• Fixed potential crash in reload logic
• system(): use string comparison instead of numeric in PID rewrite
  The meaning of the != operator has been fixed to refer to numeric comparison
  in @version: 3.8, so make sure we are using string comparison.
• Support encoding on glib compiled with libiconv
• pdbtool: Fix the ordering of the debug-info list in PatternDB
• afprog: Don't kill our own process group
  If, for some reason, the program source/destination failed to set up its
  own process group, we need to make sure we do not run killpg() on that
  process group, as it would kill ourselves.
• Handle option names with hyphen (-) characters in java scls
• dnscache performance improved
  Instead of getting rid off the per-thread DNSCache when a worker thread
  exits, store them in a linked list and acquire them as a new thread starts.
  The set of cached hostnames are valuable as worker threads come and
  go (they exit after 10seconds of inactivity), but without this
  reusing of cache instances, our DNS cache is filled again and again.
• Fixed IPv6 parser in patterndb.
• Fixed journald program name flapping
• Fixed create-dirs() inheritance in file destinations
• Fixed pass-unix-credentials() global inheritance in afunix
  The global pass-unix-credentials option was not inherited in afunix-source
  if the options{}; block was positioned lower in the
  configuration file than the given module declaration.
• Fixed create-dirs() global inheritance in afunix
  When the global create-dirs option was set to yes, the local one was ignored.
• Fixed byteorder handling on bigendian systems in netmask6 filter
• Fixed flow-control issue when overflow queue is full
  (suspending source by setting the window size to 0).
• Log HTTP response error codes in HTTPDestination (Java).
• Fixed potential leaks related $(sanitize) argument parsing in basicfuncs.
• Fixed a memory leak in python debugger
• Fixed a use-after-free bug in templates.
• Fixed a memory leak around reload in netmask6 filter.
• Fixed a memory leak in LogProtoBufferedServer in case the
  encoding() option is used.
• configure: don't override $enable_python while executing pkg-config
• Fixed BSD timestamp parsing in syslog-format.
• Fixed a SIGPIPE bug in program destination.
• Error handling has been improved in AMQP destination.
• value-pairs performance improvements, memleak fixes
• Various issues around UTF-8 support fixed.
• Fixed integer overflow in numerical operations template function
• Fixed an integer underflow in afsocket.
• Fixed numerical comperisons issues around filters.
  There's a problem in straight fixing this issue though: anyone who used
  the numeric operators erroneously will have their behaviour changed, therefore
  this patch also adds a configuration update warning in case
  someone is using the wrong syntax.
• Fixed kernel log message time drift on Linux.
• Take CRLF sequences equivalent to an LF in patterndb.
  Windows logs contain embedded CRLFs which is difficult to match against
  from db-parser(), as we use a UNIX text file to store the patterns. Also,
  the fact that the input contains CRLF whereas our patterns only contain
  an LF makes it a very unintuitive non-match, which is difficult to debug.
• When syslog-ng failed to insert data into Redis, it has crashed.
• When device file is set as a file destination then syslog-ng will not try
  to change the permission of the device file.
• Various fixes around config file parsing:
  • in some circumstances syslog-ng crashed when the config
    file contained non-readable characters
  • fixed a memory leak
  • fixed memory leak around backtick substitution

Notes to the Developers

• copyright cleanup in source tree
• install tools and scl under a syslog-ng specific subdirectory
  These should never be installed in /usr/share directly, but rather under a
  subdirectory and as described in
  https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
  we should do that right within the source and not rely on packaging tools
  to do it for us.This will trigger a required change in packaging scripts to
  avoid changing the --datadir, as the default of
  /usr/share should work out-of-the-box.
• Support for native-lanugage (compiled languages, like Rust) bindings.
  These bindings just forward the calls to the native side.
  This whole module compiles into a static library
  (libsyslog-ng-native-connector.a) which is linked to all external native
  modules. A native module defines the required functions
  (like native_parser_proxy_new()) so those symbols will be resolved.
  Some symbols have the visibility(hidden) attribute applied to them. Those
  symbols are defined by the other half of the native bindings, we only need
  their signature here. They are hidden because their definition is contained
  in other source files but we would like to keep them as "library private".
  If they are exported, the dynamic linker will resolve them when a module is
  loaded, therefore every module would be mapped to the first loaded one.
  It is best to hide everything in this native connector.
• Support system librabbitmq-c (AMQP destination)
  Currently only the internal librabbitmq-c is supported,
  if we want to use the preexisting library, the configuration
  will fail.
  This change is required if we want to get rid of the
  internal libraries.
• Added check-valgrind target to Makefile
• Remove dependency on Gradle for building Java language bindings
  (not modules, just the language binding)
• Experimental CMake support added
• Experimental OSX support added
• Improved Travis build matrix
• Added plugin skeleton creator.
• Debian packaging improved: Debian packaging from unofficial
  OBS repository has been ported to upstream.
  From now, someone could easily build debian packages from upstream source.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Adam Istvan Mozes, Andras Mitzki, Arnaud Vamorec, Balazs Scheidler,
David Schweikert, Fabien Wernli, Flavio Medeiros, Hanno Böck,
Henrik Grindal Bakken, Gergo Nagy, Gyorgy Pasztor, Laszlo Budai,
Laszlo Varady, Marc Falzon, Noemi Vanyi, Peter Czanik, Robert Fekete,
Tamas Nagy,Tibor Benke, Viktor Juhasz, Viktor Tusa, Vincent Bernat,
Zdenek Styblik, Zoltan Fried, Zoltan Pallagi, Yilin Li

3.8.0beta2

This is the second beta release for the 3.8.x series.

Changes compared to 3.8 (created by manually):

• add-contextual-data : more strict CSV parsing
  When a line in the CSV file containing more fields then required, syslog-ng won't start.
• add-contextual-data : in some circumstances syslog-ng crashed when the
  CSV file contained invalid data
• MacOSX support added to travis.yml
• FreeBSD 10.3 build issues fixed
• Oracle Solaris 11 build issues fixed
• logmsg serialization performance enhanced
• elastic-v2 and mode http added to the syslog-ng-mod-elastic Debian package
• ElasticSearch-v2: fixed missing 'path.home' issue

Changes compared to 3.7.x (automatically generated):

Note, that for beta release we generate the changes with
a tool. Final changelog will be more sophisticated (and will
include Credits section).

Bug Fixes

• Minor problems in kv-parser() Link László Várady Tamas Nagy
• Deprecate old configuration parameters in MongoDB destination Link Tibor Benke Tamas Nagy
• The output of pdbtool is scrambled Link László Várady László Várady
• @ in format json is absorbed Link pzoleex László Várady
• jni.h: No such file or directory Link czanik László Várady
• IPv6 Pattern Parser and trailing colons Link aslothinasuit László Várady
• 3.8 journal source problem Link czanik Tibor Benke
• Duplicate symbols (_TLSSslOptions and _last_parser) break build of syslog-ng 3.7.2 on OS X Link Douglas Carmichael Tamas Nagy
• Deadlock in redis destination Link suuuper Tibor Benke
• Eliminate compiler warnings and turn on -Werr Link Tibor Benke Tamas Nagy
• Deadlock with suppress option Link symphorien Balazs Scheidler

Enhancements

• support multiple drivers with the same name in syslog-ng config Link Budai Laszlo MÓZES Ádám István
• Support an alternative build system: CMake Link Tibor Benke Tibor Benke
• Verify that CFLAGS is propagated down to all of our sources Link Tamas Nagy Tamas Nagy
• Support encoding on glib compiled with libiconv Link Tamas Nagy Tamas Nagy
• Separator option for key-value parser Link Fabien Wernli Balazs Scheidler
• Copyright policy/checker Link Balazs Scheidler Tamas Nagy
• SCL for Loggly format and destination Link Robert Fekete Balazs Scheidler
• Please do not chown/chmod if the log file is a device like e.g. /dev/null Link Axel Beckert Zoltán FRIED

Fixed Issues

• Certain combination of configuration will sometimes generate incorrect unparseable JSON errors Link Allan Crooks Balazs Scheidler
• basicfuncs unit test seems to be buggy on 32bit systems Link Budai Laszlo László Várady
• Null pointer dereference when parsing malformed config Link Agostino Sarubbo Balazs Scheidler
• Aborting during configuration loading Link Tibor Benke Balazs Scheidler
• With an tag in patterndb.xml db-parser did not parse Link mitzkia Balazs Scheidler
• flapping ${PROGRAM} using systemd Link Fabien Wernli Tibor Benke
• Fix netmask6 filter on bigendian systems Link Zoltán FRIED Tibor Benke
• SIGABRT: using (") around macros in format-json type-hinting Link mitzkia Tibor Benke
• environment variables not passed on to gradle in 3.7.1 Link czanik Tibor Benke
• double-free because of the yydestruct Link Balazs Scheidler Tibor Benke
• rewrite rules don't honor time-zone Link Fabien Wernli Tibor Benke
• Integer overflow in numerical operations template function Link Fabien Wernli Tibor Benke
• distribute config.h Link Tibor Benke Tibor Benke
• lib/filter/filter-cmp.c:116: missing break ? Link dcb314 Balazs Scheidler
• plugin loading anomalies Link Tibor Benke Tibor Benke
• sigsegv in redis module when restart redis server Link Zoltán FRIED Zoltán FRIED
• 3.8 beta1 fails to compile on FreeBSD Link czanik Budai Laszlo
• Absence of clock_gettime() on OS X breaks build of lib/timeutils.c on 3.7.2 Link Douglas Carmichael László Várady
• mongodb: moving to official C mongo client library Link Budai Laszlo bkil-syslogng

Merged Pull Requests

• 'create-libsyslog-ng-so-symlink-with-cmake' : Merge branch 'create-libsyslog-ng-so-symlink-with-cmake' of git://github.com/ihrwein/syslog-ng Link MÓZES Ádám István
• 'f/fix-spelling-issues' : Merge branch 'f/fix-spelling-issues' Link Balazs Scheidler
• afstreams: include missing header (errno.h) Link Budai Laszlo
• F/freebsd10 java support Link Budai Laszlo
• elasticsearch_v2: set default path.home for node client Link Budai Laszlo
• add-contextual-data: build fixed on SLES-11 Link Budai Laszlo
• elastic-v2: fixed debian packaging Link Budai Laszlo
• FreeBSD compatibility of 3.8 - fixes #1138 Link Noémi Ványi
• F/logmsg serialize performance Link Balazs Scheidler
• Enable compatibility of make check on Mac OS X in Travis Link Tamas Nagy
• add-contextual-data: fixed double free bug when csv file contains inv… Link Budai Laszlo
• lib/Makefile.am: fixed ivykis CFLAGS Link Budai Laszlo
• add-context-data: stop when too many fields are in a line Link Noémi Ványi
• Release/3.8.0beta1 Link Budai Laszlo
• plugin skeleton creator Link Noémi Ványi
• 32bit wide NVHandle with unittests Link Noémi Ványi
• F/add contextual data Link Budai Laszlo
• Replaced the old selinux policy installer script with a new one Link Janos SZIGETVARI
• Fix segmentation fault when including SCLs Link Tibor Benke
• debian: port Debian packaging from my OBS repository Link Budai Laszlo
• F/unset value Link Balazs Scheidler
• Fix relative path canonicalization in check_java_support.m4 Link László Várady
• parser: fix invalid pointer usage in log_parser_process_message() Link László Várady
• F/fix test kv parser va arg portability Link Tamas Nagy
• Native parser: fail-fast when configuration setting failed Link Tibor Benke
• logmsg: add log_msg_new_local() function Link Tibor Benke
• include geoip tests only when geoip is enabled Link Noémi Ványi
• java-modules: gradle: removed unnecessary dependency Link Budai Laszlo
• Implement deinit() for native parsers Link Tibor Benke
• Fix invalid handling of UTF-8 strings Link László Várady
• logmpx: remove last_delivery logic Link Balazs Scheidler
• F/elasticsearch v2 mode http Link Budai Laszlo
• logmsg: don't release "original" from the cleared message Link Balazs Scheidler
• java-modules: use local maven repository instead of flat dir Link Budai Laszlo
• Remove unused syslog-ng.conf file Link László Várady
• java-modules: removed gradle cache property file Link Budai Laszlo
• Add ability to use templates in both url and message format Link avcbvamorec
• F/fix make func-test typos that keeps us from enabling it in CI Link Tamas Nagy
• Add missing free to native parser's grammar Link Tibor Benke
• Fix LL_IDENTIFIER double frees in the grammar Link Tibor Benke
• Quickstart section Link Balazs Scheidler
• F/ctx template funcs2 - mean operator, further stylistic improvements to #1037 Link Tamas Nagy
• F/dbparser pdbload refact Link Budai Laszlo
• Fix a linker related problem in systemd-journal's tests Link Tibor Benke
• F/dbparser action create context Link Balazs Scheidler
• Fix basicfuncs tests compatibility with GLIB_MIN_VERSION Link László Várady
• Add $(sum), $(min) and $(max) template functions Link László Várady
• dbparser: add XML validation in test_patterndb Link Balazs Scheidler
• Pass a GlobalConfig pointer to native_parser_proxy_new() Link Tibor Benke
• afprog: Don't kill our own process group Link hbakken
• scl: handle option names with both hyphen(-) and underscore (-) in SCLs Link Budai Laszlo
• PatternDB: fixed fastpath of empty patterndb Link juhaszviktor
• F/remove msg error etc null sentinel.sh, run on http module Link Tamas Nagy
• elasticsearch2: added an option for skipping cluster health check Link Budai Laszlo
• split test_patterndb_rule testcase Link Balazs Scheidler
• F/remove msg error etc null sentinel Link Tamas Nagy
• F/mongodb makefile cleanup Link Balazs Scheidler
• Improve "curl" module Link LiYilin
• Fix dbparser grouping-by keywords Link László Várady
• README: Corrects and updates links Link Robert Fekete
• F/missing py license (Python, Perl and Bash scripts were missing declarations) Link Tamas Nagy
• Update native parser bindings Link Tibor Benke
• Prepare OS X support Link László Várady
• service-management: if previous code was compiled with ENABLE_SYSTEMD, Link juhaszviktor
• F/file source reopen on error Link Balazs Scheidler
• elasticsearch: Added 2.2 support Link pzoleex
• F/file perms extract from cfg Link Balazs Scheidler
• Fix defines in diskq Link Tibor Benke
• DNS cache refactor and recycling Link Balazs Scheidler
• README: fix release download link Link Tibor Benke
• Fix automake warnings in native and diskq modules Link Tibor Benke
• Fix disabling MongoDB destination support in syslog-ng 3.8 Link Tibor Benke
• Fix Java's internal logger Link László Várady
• Minor linker/include fixes related to OS X build Link László Várady
• Fix diskq linking Link Tibor Benke
• Add support of Kafka 0.9.0.0 Link Tibor Benke
• java-modules.Makefile: do not copy the jar files from .gradle dir to … Link pzoleex
• systemd-journal: fix segmentation fault in g_module_symbol() on startup Link Balazs Scheidler
• Update debian example configuration file Link Shivam Mamgain
• Export GRADLE_OPTS environment variable in java-modules so Gradle can use it Link Tibor Benke
• Build Java language bindings without Gradle Link Tibor Benke
• Add Elasticsearch 2 destination with Shield support Link Budai Laszlo
• F/diskq Link juhaszviktor
• Don't fail during cleanup if there are no copyright-* logs Link Tibor Benke
• Enable -Wshadow compiler flag Link Balazs Scheidler
• Check copyright headers during the execution of make check Link Tamas Nagy
• make sure all path/dir variables are defined in a single location, by configure.ac Link Balazs Scheidler
• configure.ac: link against sasl2 lib only when it is available Link Budai Laszlo
• Fix create-dirs() inheritance in file destinations Link László Várady
• Fix java version check ('test' has '=', not '==' as operator) Link David Schweikert
• Fix global option inheritance in unix socket sources Link László Várady
• Add --keep-going to make in Travis builds Link MÓZES Ádám István
• Improve the performance of value-pairs Link Balazs Scheidler
• Implement serialization of log messages Link juhaszviktor
• Fix possible deadlock in patterndb Link juhaszviktor
• Clean up configure directory Link Balazs Scheidler
• Improve the performance of utf8-utils by using an optimized implementation of strchr() Link Balazs Scheidler
• Log HTTP response error codes in HTTP destination Link László Várady
• Use official MongoDB C Driver instead of libmongo-client Link Tamas Nagy
• Support native Elasticsearch configuration for transport mode Link Budai Laszlo
• Add inherit-environment() option to program driver Link László Várady
• Suspend source if queue is full Link juhaszviktor
• Remove fix relative path of syslog-ng in func test Link Balazs Scheidler
• Fix program driver keep-alive() option linkage Link László Várady
• Refactor misc functions Link Balazs Scheidler
• Format CEF extension Link Tamas Nagy
• F/java modpath cleanups Link Balazs Scheidler
• functest: bump config version numbers to 3.8 Link Balazs Scheidler
• macros: remove lurking semicolon after if statement Link Tibor Benke
• .travis.yml: unset PYTHON_CFLAGS Link Tamas Nagy
• templates: clean up $SOURCEIP macro expansion Link Balazs Scheidler
• F/allow new keywords with old config versions rebased to master Link Budai Laszlo
• F/unit test cleanups Link Balazs Scheidler
• F/native parser Link Tibor Benke
• F/afsql warning fixes rebased to master Link Budai Laszlo
• afstreams: get rid of superfluous conditional compilation Link Tibor Benke
• Fix compile error in Travis Link Tibor Benke
• F/groupingby parser Link Balazs Scheidler
• syslog-format: fix BSD timestamp parsing Link Tibor Benke
• improve error handling in amqp Link Balazs Scheidler
• Fix pylint errors Link Tibor Benke
• cfg-lex.l: don't report yy_fatal_error() as an unused symbol Link Balazs Scheidler
• scl: add logmatic() destination Link Balazs Scheidler
• afamqp: support systemwide librabbitmq-c Link MÓZES Ádám István
• F/value pairs cleanups Link Balazs Scheidler
• scl: add apache-accesslog-parser() Link Balazs Scheidler
• F/dbparser multiline patterns Link Balazs Scheidler
• F/minor fixes Link Balazs Scheidler
• F/date parser stamp Link Balazs Scheidler
• Fix warning messages in afprog Link MÓZES Ádám István
• linux-audit-scanner: recognize a0-a9* as fields to be decoded Link Balazs Scheidler
• F/whitespace fixes Link Balazs Scheidler
• Fix an integer underflow. Link Hanno Böck
• func_test: fix up test_python Link Balazs Scheidler
• csvparser refactor, prefix() and dialect() options Link Balazs Scheidler
• port date-parser() from incubator Link Balazs Scheidler
• F/opening 3.8 branch Link Budai Laszlo
• Keep the program destination open between configuration reloads Link MÓZES Ádám István
• contrib/syslog-debun: portability improvements and fixes Link PÁSZTOR György
• logmsg: Fix a logmsg_cached_abort assignment Link MÓZES Ádám István
• syslog-ng.pc: add eventlog as a required package Link Tibor Benke

3.8.0beta1

This is the first beta release for the 3.8.x series.

Changes compared to 3.7.x:
Note, that for beta release we generate the changes with
a tool (may contain false information). Final changelog will be more sophisticated (and will
include Credits section).

Implemented enhancements:

• Support an alternative build system: CMake #966
• SCL for Logmatic format and destination #799
• SCL for Loggly format and destination #798
• support multiple drivers with the same name in syslog-ng config #661
• HTTP destination driver in Java #539
• HTTP destination driver in Python #534
• F/unset value #1108 (bazsi)
• F/elasticsearch v2 mode http #1053 (lbudai)
• Add $(sum), $(min) and $(max) template functions #1037 (MrAnno)
• Add ability to use templates in both url and message format #1033 (avcbvamorec)
• F/libmongo client compatibility over mongo c driver #981 (bkil-syslogng)
• Improve "curl" module #978 (litterbear)
• Prepare OS X support #953 (MrAnno)
• Add Elasticsearch 2 destination with Shield support #912 (lbudai)
• Use official MongoDB C Driver instead of libmongo-client #891 (bkil-syslogng)
• Support native Elasticsearch configuration for transport mode #890 (lbudai)
• Set 0.11.0 as the minimal required version of hiredis to avoid possible deadlocks #887 (ihrwein)
• Add inherit-environment() option to program driver #861 (MrAnno)
• Remove fix relative path of syslog-ng in func test #858 (bazsi)
• Add support of Kafka 0.9.0.0 #856 (ihrwein)
• Log HTTP response error codes in HTTP destination #855 (MrAnno)
• Improve the performance of value-pairs #851 (bazsi)
• Format CEF extension #842 (bkil-syslogng)
• Implement serialization of log messages #834 (juhaszviktor)
• scl: add logmatic() destination #812 (bazsi)
• F/scl varargs refined #699 (ihrwein)
• F/unix socket source creates dir #632 (ihrwein)
• ... NEWS.md

Fixed bugs:

• The output of pdbtool is scrambled #1043
• 3.8 journal source problem #914
• Global option inheritance problem in afunix-source #894
• Deadlock in redis destination #792
• Deadlock with suppress option #781
• tests/unit/test_zone fails on Unix epoch #726
• Every second config reload kills marking #701
• Runs in a different $CWD when foregrounding via "-F" #700
• Segfault on TLS errors #695
• Compile error related to python module #674
• syslog-ng is stuck in an infinite loop of setsockopt() returning ENOTSOCK #670
• syslog-ng 3.6 may kill init process #586
• message formatting on remote destinations did not follow the switch to legacy from IETF syslog format #570
• Missing mark message on TCP destination in case of mark_mode(dst_idle) #547
• Cannot write filter plugins #427
• ... NEWS.md

Unofficial Debian packages:

• https://build.opensuse.org/project/show/home:laszlo_budai:syslog-ng

3.7.3

Changes compared to 3.7.2:

Improvements

• Updated Python package requirements.
• Can now compile without MongoDB.
• Added eventlog to the list of required pkg-config packages.
• Basic FreeBSD and HP-UX support of syslog debug bundle generator by
  improving POSIX shell compatibility.
• Keep the program destination open between configuration reloads.
• system-source now uses keep-timestamp(no) for Linux kernel log.
  The time source used by /dev/kmsg is not updated after system
  SUSPEND/RESUME.

Fixes

• Fix a SIGSEGV when a Redis command returns an error.
• Resolve deadlock in logwriter triggered by suppress()
• Mitigate possible deadlock in patterndb
• Fixed global inheritance of pass-unix-credentials() and create-dirs().
• Certain compilers complained about an undefined symbol when setting
  keep-alive(yes).
• For certain use cases, afsocket would not handle procfs read errors due
  to an integer underflow.
• Enhanced Java version check and the handling of SyslogNgInternalLogger
  (used by Kafka), the FATAL loglevel and getLocationInformation().
• When a big amount of kernel log was produced in a very short time,
  the syslog-ng process sometimes entered into a spin and stop processing
  messages.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

András Mitzki, Avleen Vig, Balázs Scheidler, Ben Kibbey, Christian Herzig,
David Schweikert, Douglas Carmichael, Dezso Endre Molnar, Fabien Wernli,
Gergely Czuczy, Gergely Nagy, Gergo Nagy, Hanno Böck, Herzig, Christian,
Laszlo Budai, László Várady, MÓZES Ádám István, PÁSZTOR György, Péter Czanik,
Robert Fekete, Saurabh Shukla, Tamás Nagy, Tibor Benke, Viktor Juhász,
Vincent Bernat, Wang Long, Zdenek Styblik, Zoltán FRIED, Zoltán Pallagi

3.7.2

This is the first maintenance release for the 3.7.x series.

Changes compared to 3.7.1:

Improvements

• Added mbox() source.
  This source can be used to fetch emails from local mbox files:
  source { mbox("/var/spool/mail/root"); };
  This will fetch root emails and parse them into a multiline $MSG.
  Original implementation by Fabien Wernli, I only converted it into
  an SCL.
• It is possible to append dynamically options into SCL blocks from now.
• concurrent_request option added to ElasticSearch with default value 1.
• In elasticsearch destinaton, message_template() argument renamed to
  template().
• SCL added to every Java module (ElasticSearch, Kafka, HDFS).
• Linux Audit Parser added for parsing key-value pairs produced by
  the Linux Audit subsystem.
• HTTP destination is now able to receive HTTP method as an option.
  All the supported methods are available
  (POST, PUT, HEAD, OPTIONS, DELETE, TRACE, GET).

Fixes

• In some circumstances syslog-ng mod-journal re-read every already
  processed messages.
• When syslog-ng got a reload and the reload process done within 1 second then
  mafter the reload, syslog-ng stop generating mark-messages.
• When initialization of a network destination in syslog-ng failed (eg. due to
  DNS resolution failure) we didn't create a queue which caused message loss.
• syslog-ng segfaulted on TLS errors when wrong certs was provided
  (eg.: CA cert with the cert-file directive instead of the server cert).
• Fixed a continuous spinning case in the file driver, when the
  destination file is a device (e.g. /dev/stdout).
• A memory leak in around template functions in grammar fixed.
• Fixed Python3 support.
• Fixed Python GIL issue in python destination.
• From now, instead of skipping doc/ alltogether when ENABLE_MANPAGES is
  not set, only skip the actual man pages, but handle the rest properly.
• Allow overriding the python setup.py options.

When installing the python modules, allow overriding the options. This
is useful for distributions that want to pass extra options. For
example, on Debian, we want --install-layout="deb" instead of the
--prefix and --root options.

With this change, the previous behaviour remains the default, but one
can supply PYSETUP_OPTIONS on the make command-line to override it.

• The systemd service file read /etc/default/syslog-ng and /etc/sysconfig/syslog-ng,
  but didn't do anything with their contents. $SYSLOGNG_OPTS added to ExecStart, so
  that the EnvironmentFiles have an effect (at least on Debian).
• Java support checking fixed (not only jdk is required but also gradle).
• Memory leak around ping() in Redis fixed.
• A crash in pdbtool fixed around r_parser_email().
• Removed cygwin fdlimit statement.
  Make the default for RLIMIT_NOFILE equal to the current system limits.
  --fd-limit can still override this, but the default will be configured
  based on existing system limits.
• Fixed BSD year inference.
  Fixed logic and made clearer the inference of year from bsd-style
  rfc3164 syslog-messages, which do not include a year.
• Handle correctly the epoch 0 timestamp.
  (Previously, syslog-ng cached the zero timestamp and treated 1970 as it was
  1900.)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Adam Arsenault, Adam Istvan Mozes, Andras Mitzki, Avleen Vig,
Balazs Scheidler, Fabien Wernli, Gergely Czuczy, Gergely Nagy, Gergo Nagy,
Laszlo Budai, Peter Czanik, Robert Fekete, Saurabh Shukla, Tamas Nagy,
Tibor Benke, Viktor Juhasz, Vincent Bernat, Wang Long, Zdenek Styblik,
Zoltan Pallagi.

3.7.1

New dependencies

OpenSSL is now a required dependency for syslog-ng because the newly added
hostid and uniqid features requires a CPRNG provided by OpenSSL.
Therefore non-embedded crypto lib is not a real option, so the support of
having such a crypto lib discontinued and all SSL-dependent features enabled
by default.

Library updates

• Minimal libriemann-client version bumped from 1.0.0 to 1.6.0.
• Added support for the monolithic libsystemd library (systemd 209).
• RabbitMQ submodule upgraded.

Features

Language bindings

• Java-destination driver ported from syslog-ng-incubator.
  Purpose of having Java destination driver is to make it possible
  to implement destination drivers in the Java language (and using
  'official' Java client libraries).
• Python language support is ported from syslog-ng incubator and
  has been completely reworked. Now, it is possible to implement template
  functions in Python language and also destination drivers.
  Main purpose of supporting Python language is to implement a nice
  interactive syslog-ng config debugger for syslog-ng.

New drivers

New Java destination drivers

ElastiSearch, Kafka and HDFS destination drivers are implemented by using
the 'official' Java client libraries and syslog-ng provides a way to set
their own, native configuration file. Log messages generated by the client
Java libraries are redirected to syslog-ng via our own Log4JAppender which
means that those logs are available as internal syslog-ng messages.

• ElasticSearch
• Kafka
• Hadoop/HDFS
• HTTP

Parsers

• Added a geoip() parser, that can look up the country code and
  latitude/longitude information from an IPv4 address. For lat/long to
  work, one will need the City database.
• New parser, extract-solaris-msgid() added for automatically extracts
  (parses & removes) the msgid portion of Solaris messages.
• Extended the set of supported characters to every printable ASCII's except
  ., [ and ] in extract-prefix for json-parser().
• Added string-delimiters option to csvparser to support multi character
  delimiters in CSV parsing.
• A kv-parser() introduced for WELF (WebTrens Enhanced Log Format) that
  implements key=value parsing. The kv-parser() tries to extract
  key=value formatted name-value pairs from the input string.
• value-pairs: make it possible to pass --key as a positional argument
  From now it is possible to use value-pairs expressions like this:
  $(format-json MSG DATE)
  instead of
  $(format-json --key MSG --key DATE)

Filters

• Added IPv6 netmask filter for selecting only messages sent by a host whose
  IP address belongs to the specified IPv6 subnet.

Macros

• Added a new macro, called HOSTID which is a 32-bit number generated by
  a cryptographically secure PRNG. Its purpose is to identify the
  syslog-ng host, thus it is the same for every message generated on the same
  host.
• Added a new macro, called UNIQID which is a practically unique ID generated
  from the HOSTID and the RCPTID in the format of HOSTID@RCPTID.
  Uniqid is a derived value: it is built up from the always available hostid
  and the optional rcptid. In other words: uniqid is an extension over rcptid.
  For that reason use-rcptid has been deprecated and use-uniqid could be
  use instead.

Templates

• welf was renamed to kvformat
  As this reflects the purpose of this module much better, WELF is just
  one of the format it has support for.
• $(format-cim) template function added into an SCL module.
• It is possible to create templates without braces.

SMTP destination

• The afsmtp driver now supports templatable recipients field.
  Just like the subject() and body() fields, now the address containing
  parameters of to(), from(), cc() and bcc() can contain macros.

Unix Domain Sockets

• Added pass-unix-credentials() global option for enabling/disabling unix
  credentials passing on those platforms which has this feature. By default
  it is enabled.
• Added create-dirs() option to unix-*() sources for creating the
  containing directories for Unix domain sockets.

Riemann destination

• Added batched event sending support for riemann destination driver which
  makes the riemann destination respect flush-lines(), and send event
  in batches of configurable amount (defaults to 1). In case of an error,
  all messages within the batch will be dropped. Dropped messages, and
  messages that result in formatting errors do not count towards the batch
  size. There is no timeout, but messages will be flushed upon deinit.
• A timeout() option added to the Riemann destination.

PatternDB

• Earlier, in patterndb, the first applicable rule won, even if it was
  only a partial match. This means that when rules overlapped, the shorter
  match would have been found, if it was the first to be loaded.
  A strong preference introduced for rules that match the input string
  completely. The load order is still applicable though, it is possible to
  create two distinct rules that would match the same input, in those cases
  the first one to be loaded wins.

Miscellaneous features

• New builtin interactive syslog-ng.conf debugger implemented for syslog-ng.
  The debugger has a Python frontend which contains a full Completer
  (just press TABs and works like bash)
• Added a reset option to syslog-ng-ctl stats. With this option the non-stored
  stats counters can be zeroed.
• New parameter added to loggen: --permanent (-T) wich is for sending logs
  indefinitely.
• Loggen uses the proper timezone offset in generated message.
• The ssl_options inside tls() extended with the following set:
  no-sslv2, no-sslv3, no-tlsv1, no-tlsv11, no-tlsv12.
• Added syslog-debug bundle generator script to make it easier to reproduce bugs
  by collecting debug related information, like:
  • process information gathering
  • syscall tracing (strace/truss)
  • configuration gathering
  • selinux related information gathering
  • solaris information gathering (sysdef, kstat, showrev, release)
  • get information about syslog-ng svr4 solaris packages, if possible

Bugfixes

• New utf8 string sanitizers instead of old broken one.

• syslog-ng won't send SIGTERM when getpgid() fails in program destination
  (afprog).

• In some cases program destination respawned during syslog-ng stop/restart
  (afprog).

• syslog-ng generates mark messages when mark-mode is set
  to host-idle.

• Using msg_control only when credential passing is supported in socket
  destination (afsocket).

• Writer is replaced only when protocol changed during reload in socket
  destination (afsocket).

• Fix spinning on EOF for unix-stream() sockets. Root cause of the spinning
  was that a unix-dgram socket was created even in case of unix-stream.

• When the configured host was not available during the initialization of
  afsocket destination syslog-ng just didn't start. From now, syslog-ng
  starts in that case and will retry connecting to the host periodically.

• Fixed BSD year inference in syslogformat. When the difference between the
  current month and the month part of the timestamp of an incoming logmessage
  in BSD format (which has no year part) was greater than 1 then syslog-ng
  computed the year badly.

• In some cases, localtime related macros had a wrong value(eg.:$YEAR).

• TLS support added to Riemann destination

• Excluded "tags" from Riemann destination driver as an attribute which
  conflicts with reserved keyword

• When a not writeable/non-existent file becomes writeable/exists later,
  syslog-ng recognize it (with the help of reopen-timer) and delivers messages
  to the file without dropping those which were received while the file was
  not available (affile).

• Fixed a crash around affile at the first message delivery when templates
  were used (affile).

• Fixed a configure error around libsystemd-journal.

• Removed syslog.socket from service file on systems using systemd.
  Syslog-ng reads the messages directly from journal on systems with systemd.

• Fixed compilation where the monolitic libsystemd was not available.

• Fixed compilation failure on OpenBSD.

• AMQP connection process fixed.

• Added DOS/Windows line ending support in config.

• Retries fixed in SQL destination. In some circumstances when
  retry_sql_inserts was set to 1, after an insertion failure all incoming
  messages were dropped.

• Transaction handling fixed in SQL destination. In some circumstances when
  both select and insert commands were run within a single transaction and
  the select failed (eg.: in case of mssql), the log messages related to
  the insert commands, broken by the invalid transaction, were lost.

• Fixed a memleak in SQL destination driver.
  The memleak occured during one of the transaction failures.

• Memory leak around reload and internal queueing mechanism has been fixed.

• Fixed a potential abort when the localhost name cannot be detected.

• Security issue fixed around $HOST.
  Tech details:
  When the name of the host is too long, the buffer we use to format the
  chained hostname is truncated. However snprintf() returns the length the
  result would be if no truncation happened, thus we will read uninitialized
  bytes off the stack when we use that pointer to set $HOST
  with log_msg_set_value().

  There can be some security implications, like reading values from the stack
  that can help to craft further exploits, especially in the presense of
  address space randomization. It can also cause a DoS if the hostname length
  is soo large that we would read over the top-of-the-stack, which is probably
  not mmapped causing a SIGSEGV.

• Journal entries containing name-value pairs without '=' caused syslog-ng
  to crash. Instead of crashing, syslog-ng just drop these nv pairs.

• Fixed the encoding of characters below 32 if escaping is enabled in
  templates. Templated outputs never contained references to characters below
  32, essentially they were dropped from the output for two reasons:

  • the prefixing backslash was removed from the code
  • the format_uint32_padded() function produced no outputs in base 8

• Fixed afstomp destination port issue. It always tried to connect to the port 0.

• Fixed memleak in db-parser which could happen at every reload.

• Fixed a class of rule conflicts in db-parser:
  Because an error in the pdb load algorithms, some rules would conflict which
  shouldn't have done that. The problem was that several programs would use
  the same RADIX tree to store their patterns. Merging independent programs
  meant that if they the same pattern listed, it would clash, even though
  their $PROGRAM is different.

  There were multiple issues:

  • we looked up pattern string directly, even they might have contained
    @parser@ references. It was simply not designed that way and only
    worked as long as we didn't have the possibility to use parsers
    in program names
  • we could merge programs with the same prefix, e.g.
    su, supervise/syslog-ng and supervise/logindexd would clash, on "su",
    which is a common prefix for all three.

  The solution involved in using a separate hash table for loading, which
  at the end is turned into the radix tree.

• pdbtool match when used with the --debug-pattern option used a low-level
  lookup function, that didn't perform all the db-parser actions specified
  in the rule

• Max packet length for spoof source is set to 1024 (previously : 256).

• A certificate which is not contained by the list of fingerprints is
  rejected from now.

• Hostname check in tls certificate is case insensitive from now.

• There is a use-case where user wants to ignore an assignment to a name-value
  pair. (eg.: when using csv-parser(), sometimes we get a column we really
  want to drop instead of adding it to the message). In previous versions an
  error message was printed out:
  'Name-value pairs cannot have a zero-length name'.
  That error message has been removed.

• Fixed a docbook related compilation error: there was a hardcoded path that
  caused build to fail if docbook is not on that path. Debian based
  platforms did not affected by this problem.
  Now a new option was created for ./configure that is --enable-manpages
  that enables the generation of manpages using docbook from online source.
  '--with-docbook=PATH' gives you the opportunity to specify the path for
  your own installed docbook.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Adam Arsenault, Adam Istvan Mozes, Alex Badics, Andras Mitzki,
Balazs Scheidler, Bence Tamas Gedai, Ben Kibbey, Botond Borsits, Fabien Wernli,
Gergely Nagy, Gergo Nagy, Gyorgy Pasztor, Kristof Havasi, Laszlo Budai,
Manikandan-Selvaganesh, Michael Sterrett, Peter Czanik, Robert Fekete,
Sean Hussey, Tibor Benke, Toralf Förster, Viktor Juhasz, Viktor Tusa,
Vincent Bernat, Zdenek Styblik, Zoltan Fried, Zoltan Pallagi.

3.7.0beta2

This is the second beta release of the upcoming syslog-ng OSE 3.7
branch.

Changes compared to the previous alpha release:

Features

• Added a geoip parser.
• ssl_options inside tls() extended with the following set:
  no-sslv2, no-sslv3, no-tlsv1, no-tlsv11, no-tlsv12
• minimal libriemann-client version bumped from 1.0.0 to 1.6.0
• TLS support added to Riemann destination
• timeout() option added to Riemann destination

Fixes

• SyslogNg.jar removed from the release tarball.
• When the configured host was not available during the initialization of
  afsocket destination syslog-ng just didn't start. From now, syslog-ng
  starts in that case and will retry connecting to the host periodically.
• When a not writeable file becomes writeable later, syslog-ng recognize it
  (with the help of reopen-timer) and delivers messages to the file without
  dropping those which were received during the file was not available.
• Fixed a configure error around libsystemd-journal.
• --disable-python option and other Python related fixes addded to
  configure
• Retries fixed in SQL destination. In some circumstances when
  retry_sql_inserts was set to 1, after an insertion failure all incoming
  messages were dropped.
• Added DOS/Windows line ending support in config.
• Parallel build is supported for Python and Java destination drivers.
• Fixed compilation failure on OpenBSD
• Memory leak around reload and internal queueing mechanism has been fixed.
• AMQP connection process fixed.
• Fixed a potential abort when the localhost name cannot be detected.
• Security issue fixed around $HOST.
  Tech details:
  When the name of the host is too long, the buffer we use to format the
  chained hostname is truncated. However snprintf() returns the length the
  result would be if no truncation happened, thus we will read uninitialized
  bytes off the stack when we use that pointer to set $HOST
  with log_msg_set_value().
  There can be some security implications, like reading values from the stack
  that can help to craft further exploits, especially in the presense of
  address space randomization. It can also cause a DoS if the hostname length
  is soo large that we would read over the top-of-the-stack, which is probably
  not mmapped causing a SIGSEGV.
• Journal entries containing name-value pairs without '=' caused syslog-ng
  to crash. Instead of crashing, syslog-ng just drop these nv pairs.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Alex Badics, Andras Mitzki, Balazs Scheidler, Bence Tamas Gedai,
Fabien Wernli, Gergely Nagy, Gergo Nagy, Gyorgy Pasztor, Istvan Adam Mozes,
Laszlo Budai, Peter Czanik, Robert Fekete, Tibor Benke, Viktor Juhasz,
Zoltan Pallagi.

3.6.4

This is the fourth maintenance (extra) release for 3.6.x series
and fixes some critical issues.

Fixes

• systemd support fixed on those platforms which has systemd < 209
  (with modular libraries)
• on some platforms(eg.: RHEL6) there was a configure error around
  libsystemd-journal
• AMQP segfaulted right after starting on some platforms

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Balazs Scheidler, Laszlo Budai, Peter Czanik, Tibor Benke,
Viktor Juhasz .

3.6.3

This is the third maintanance release for 3.6.x series.

Changes compared to 3.6.2:

Core fixes

• Inaccurate timestamps fixed on Linux for messages read from /dev/kmsg.
  For those messages syslog-ng uses keep-timestamp(no).
• Added DOS/Windows line ending support in config.
• In some cases, not all the existing plugins were loaded by default.
• In some cases, syslog-ng crashed during stop phase when user wanted
  syslog-ng to stop immediately after start.
• Some memory leak around reload and internal queueing mechanism has been fixed.

Build related fixes

• Manpage build issue fixed by adding --enable-manpages and --with-docbook
  configure option. --with-docbook=PATH gives the user the opportunity to
  specify the path for the user's own installed docbook.
• Fixed parallel build by adding correct dependencies to
  syslog-ng-ctl/Makefile.am.

Module fixes

• When a not writeable file becomes writeable later, syslog-ng recognize it
  (with the help of reopen-timer) and delivers messages to the file without
  dropping those which were received during the file was not available.
• Fixed a crash at the first message delivery when templates are used in
  a filename.
• Fixed a memory leak around file destination driver.
• In some circumstances, during reload, syslog-ng crashed when
  high internal message rate occured.
• When the configured host was not available during the initialization of
  afsocket destination syslog-ng just didn't start. From now, syslog-ng
  starts in that case and will retry connecting to the host periodically.
• Retries fixed in SQL destination. In some circumstances when
  retry_sql_inserts was set to 1, after an insertion failure all incoming
  messages were dropped.
• Connection process fixed in amqp destination and RabbitMQ module is
  set to upstream.
• Monolithic libsystemd library support added.
  In systemd 209, the various small libsystemd-* libraries were merged
  into a single libsystemd. From now, syslog-ng detects and
  uses the merged library when present, while still supports the split
  ones too. If the merged library is found, that will be preferred.
• Destination port fixed in afstomp.
• A memory leak fixed around ping functionality in redis.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Adam Mozes, Andras Mitzki, Balazs Scheidler, Ben Kibbey, Fabien Wernli,
Gergely Nagy, Gergo Nagy, Henrik Grindal Bakken, Laszlo Budai, Peter Czanik,
Pradeep Sanders, Robert Fekete, Tibor Benke, Tomáš Novosad, Toralf Förster,
Viktor Juhasz, Viktor Tusa, Zoltan Pallagi .

3.7.0beta1

This is the first beta release of the upcoming syslog-ng OSE 3.7
branch.

Further releases will focus on fixes and small Getting started ...
documentations.

Changes compared to the previous alpha release:

Features

• Added batched event sending support for riemann destination driver which
  makes the riemann destination respect flush-lines(), and send event
  in batches of configurable amount (defaults to 1). In case of an error,
  all messages within the batch will be dropped. Dropped messages, and
  messages that result in formatting errors do not count towards the batch
  size. There is no timeout, but messages will be flushed upon deinit.
• Added IPv6 netmask filter for selecting only messages sent by a host whose
  IP address belongs to the specified IPv6 subnet.
• Added syslog-ng debug bundle generator script for collecting debug related
  information.
• Added a new macro, called HOSTID which is a 32-bit number generated by
  a cryptographically secure PRNG. Its purpose is to identify the
  syslog-ng host, thus it is the same for every message generated on the same
  host.
• Added a new macro, called UNIQID which is a practically unique ID generated
  from the HOSTID and the RCPTID in the format of HOSTID@RCPTID.
  Uniqid is a derived value: it is built up from the always available hostid
  and the optional rcptid. In other words: uniqid is an extension over rcptid.
  For that reason use-rcptid has been deprecated and use-uniqid could be
  use instead.
• Added a reset option to syslog-ng-ctl stats. With this option the non-stored
  stats counters can be zeroed.
• Java-destination driver ported from syslog-ng-incubator.
  Purpose of having Java destination driver is to provide the right way to
  support all player in the "Java related logging ecosystem"
  (Kafka, HDFS, ElasticSearch, ...). Java dest driver is a special driver,
  a bridge between the C and the Java world from syslog-ng point of view.
• Python language support is ported from syslog-ng incubator and
  has been completely reworked. Now, it is possible to implement template
  functions in Python language and also destination drivers.
  Main purpose of supporting Python language is to implement a nice
  interactive syslog-ng config debugger for syslog-ng.
• New builtin interactive syslog-ng.conf debugger implemented for syslog-ng.
  The debugger has a Python frontend which contains a full Completer
  (just press TABs and works like bash)

Enhancements

• Extended the set of supported characters to every printable ASCII's except
  ., [ and ] in extract-prefix for json-parser().

• OpenSSL set as a hard dependency for syslog-ng because the newly added
  hostid and uniqid features requires a CPRNG provided by OpenSSL.

  After OpenSSL is a hard dependency

  • non-embedded crypto lib is not a real option, so the support of having
    such a crypto lib discontinued
  • all SSL-dependent features enabled by default

• Added string-delimiters option to csvparser to support multi character
  delimiters in CSV parsing.

• Upgrade RabbitMQ submodule to the upstream.

• Extended rcpt-id to 64 bits (formerly it was 48 bits).

Fixes

• Fixed the encoding of characters below 32 if escaping is enabled in
  templates. Templated outputs never contained references to characters below
  32, essentially they were dropped from the output for two reasons:

  • the prefixing backslash was removed from the code
  • the format_uint32_padded() function produced no outputs in base 8

• Fixed afstomp destination port issue. It always tried to connect to the port 0.

• Fixed compilation where the monolitic libsystemd was not available.

• Fixed memleak in db-parser which could happen at every reload.

• Fixed a class of rule conflicts in db-parser:

  Because an error in the pdb load algorithms, some rules would conflict which
  shouldn't have done that. The problem was that several programs would use
  the same RADIX tree to store their patterns. Merging independent programs
  meant that if they the same pattern listed, it would clash, even though
  their $PROGRAM is different.

  There were multiple issues:

  • we looked up pattern string directly, even they might have contained
    @parser@ references. It was simply not designed that way and only
    worked as long as we didn't have the possibility to use parsers
    in program names
  • we could merge programs with the same prefix, e.g.
    su, supervise/syslog-ng and supervise/logindexd would clash, on "su",
    which is a common prefix for all three.

  The solution involved in using a separate hash table for loading, which
  at the end is turned into the radix tree.

  • Fixed a crash around affile at the first message delivery when templates
    were used.
  • Excluded "tags" from riemann destination driver as an attribute which
    conflicts with reserved keyword
  • Fixed a docbook related compilation error: there was a hardcoded path that
    caused build to fail if docbook is not on that path. Debian based
    platforms did not affected by this problem.
    Now a new option was created for ./configure that is --enable-manpages
    that enables the generation of manpages using docbook from online source.
    '--with-docbook=PATH' gives you the opportunity to specify the path for
    your own installed docbook.

Developer notes

• filter: fix external filter plugin lookup

  The filter_plugin rule expected an LL_IDENTIFIER and filter_comparison
  expected a string which in turn is an LL_IDENTIFIER or LL_STRING. It
  caused a conflict in the grammar which prevented to load external
  filter plugins.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Balazs Scheidler, Botond Borsits, Fabien Wernli, Gergely Nagy,
Gergo Nagy, Gyorgy Pasztor, Istvan Adam Mozes, Laszlo Budai,
Manikandan-Selvaganesh, Peter Czanik, Robert Fekete, Tibor Benke,
Viktor Juhasz, Vincent Bernat, Zoltan Fried, Zoltan Pallagi.

3.6.2

This is the first maintenance release for 3.6.x series.

Changes compared to 3.6.1:

Features

• New parameter added to loggen: --permanent (-T) wich is for sending logs
  indefinitely.

Fixes

• From now, syslog-ng won't crash when using a Riemann destination and
  no attributes are set.
• In some cases program destination respawned during syslog-ng stop/restart.
• Max packet length for spoof source is set to 1024 (previously : 256).
• Removed syslog.socket from service file on systems using systemd.
  Syslog-ng reads the messages directly from journal on systems with systemd.
• In some cases, localtime related macros had a wrong value(eg.:$YEAR).
• Transaction handling fixed in SQL destination. In some circumstances when
  both select and insert commands were run within a single transaction and
  the select failed (eg.: in case of mssql), the log messages related to
  the insert commands, broken by the invalid transaction, were lost.
• Fixed a memleak in SQL destination driver.
  The memleak occured during one of the transaction failures.
• A certificate which is not contained by the list of fingerprints is
  rejected from now.
• Hostname check in tls certificate is case insensitive from now.
• Fix spinning on EOF for unix-stream() sockets. Root cause of the spinning
  was that a unix-dgram socket was created even in case of unix-stream.

Compatibility notes

• Prefer SYSLOG_IDENTIFIER over _COMM in systemd-journal.
  In order to not break assumptions, prefer SYSLOG_IDENTIFIER over _COMM.
  For example, postfix uses postfix/qmgr as SYSLOG_IDENTIFIER, but _COMM
  is only "qmgr". The journal itself uses SYSLOG_IDENTIFIER when
  reconstructing the syslog message, so we should not deviate from that
  behaviour, either.

  Similarly, rsyslog also prefers SYSLOG_IDENTIFIER, so for the sake of
  compatibility, doing the same is preferable.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Alexander Görtz, Andras Mitzki, Balazs Scheidler, Fabien Wernli, Gergely Nagy,
Jasper Lievisse Adriaanse, Laszlo Budai, Michael Sterrett, Peter Czanik,
Robert Fekete, Tibor Benke, Viktor Juhasz, Viktor Tusa, Zoltan Fried .

3.7.0alpha2

This is the second alpha release of the syslog-ng OSE 3.7
branch.

Changes compared to the previous alpha release:

Features

• Added support for the monolithic libsystemd library (systemd 209).
• New parameter added to loggen: --permanent (-T) wich is for sending logs
  indefinitely.
• Earlier, in patterndb, the first applicable rule won, even if it was
  only a partial match. This means that when rules overlapped, the shorter
  match would have been found, if it was the first to be loaded.
  A strong preference introduced for rules that match the input string
  completely. The load order is still applicable though, it is possible to
  create two distinct rules that would match the same input, in those cases
  the first one to be loaded wins.
• New parser, extract-solaris-msgid() added for automatically extracts
  (parses & removes) the msgid portion of Solaris messages.

Fixes

• In some cases program destination respawned during syslog-ng stop/restart.
• Max packet length for spoof source is set to 1024 (previously : 256).
• Removed syslog.socket from service file on systems using systemd.
  Syslog-ng reads the messages directly from journal on systems with systemd.
• In some cases, localtime related macros had a wrong value(eg.:$YEAR).
• Transaction handling fixed in SQL destination. In some circumstances when
  both select and insert commands were run within a single transaction and
  the select failed (eg.: in case of mssql), the log messages related to
  the insert commands, broken by the invalid transaction, were lost.
• Fixed a memleak in SQL destination driver.
  The memleak occured during one of the transaction failures.
• A certificate which is not contained by the list of fingerprints is
  rejected from now.
• Hostname check in tls certificate is case insensitive from now.
• Fix spinning on EOF for unix-stream() sockets. Root cause of the spinning
  was that a unix-dgram() socket was created even in case of unix-stream().
• There is a use-case where user wants to ignore an assignment to a name-value
  pair. (eg.: when using csv-parser(), sometimes we get a column we really
  want to drop instead of adding it to the message). In previous versions an
  error message was printed out:
  'Name-value pairs cannot have a zero-length name'.
  That error message has been removed.
• pdbtool match when used with the --debug-pattern option used a low-level
  lookup function, that didn't perform all the db-parser actions specified
  in the rule

Developer notes

• PatternDB lookup refactored (it is easier to understand the code).

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Balazs Scheidler, Fabien Wernli, Gergely Nagy, Laszlo Budai,
Michael Sterrett, Peter Czanik, Robert Fekete, Tibor Benke, Sean Hussey,
Viktor Juhasz, Viktor Tusa, Zoltan Fried .

3.7.0alpha1

This is the first alpha release of the syslog-ng OSE 3.7
branch.

Changes compared to the latest stable release (3.6.1):

Features

• It is possible to create templates without braces.
• User defined template-function support added.
  User can define template functions in her/his configuration the same
  way she/he would define a template.
• $(format-cim) template function added into an SCL module.
• A new choice for inherit-properties implemented that will merge
  all name-value pairs into the new synthetic message, with the most recent
  being beferred over older values.

Developer notes

• Added implementation for user-defined template functions.
  A new API added, user_template_function_register() that allows
  registering a LogTemplate instance as a template function, dynamically.

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Balazs Scheidler, Fabien Wernli, Gergely Nagy, Laszlo Budai,
Peter Czanik, Viktor Juhasz, Viktor Tusa