Release v1.29.3

Release v1.29.3 is a patch release to improve the stability of the Netdata Agent.

We discovered a bug that when proc.plugin attempts to collect the operstate parameter for a virtual network interface. If the chart is obsoleted, the Netdata Agent crashes. This bug was fixed in #10667. We're grateful to @gaia for first identifying this issue and working with our engineers, along with @sdellenb, to provide logs and point us toward the source of the bug.

This release also contains additional bug fixes and improvements.

Packaging/installation

• Fixed condition controlling use of static LWS in RPM builds. (#10661, @Ferroin)

Documentation

• Improve the StatsD documentation and associated Statsd dashboard improvements. (#10640, @OdysLam)
• Fix broken links in docs and add collectors to list. (#10651, @joelhans)
• Fix wrong link on docs Netdata Agent daemon. (#10659, @OdysLam)

Bug fixes

• Fix proc.plugin to invalidate RRDSETVAR pointers on obsoletion. (#10667, @mfundul)

Release v1.29.2

Release v1.29.2 is a patch release to improve the stability of the Netdata Agent.

We discovered that an improvement introduced in v1.29.0 could inadvertently set all os_* host labels to unknown, which could affect users who leverage these host labels to organize their nodes, deploy health entities, or export metrics to external time-series databases. This bug was fixed in #10647.

This release also contains additional bug fixes and improvements.

Acknowledgments

• @tinyhammers for making the Opsgenie API URL configurable.
• @vjt for documenting the scheme option in the elastic search collector.
• @rda0 for the fix that does not allow binary data to be printed.
• @fayak for adding freeswitch to the app_groups.conf file.

Improvements

• Make the Opsgenie API URL configurable. (#10561, @tinyhammers)
• Add k8s_cluster_id host label. (#10588, @ilyam8)
• Enable apps.plugin aggregation debug messages. (#10645, @vlvkobal)
• Add configuration parameter to disable stock alarms. (#10617, @thiagoftsm)
• Add ACLK proxy setting as host label. (#10619, @underhood)
• Add freeswitch to apps_groups.conf. (#10621, @fayak)
• Simplify thread creation and remove unnecessary variables in the eBPF plugin. (#10442, @thiagoftsm)

Documentation

• Fix a typo in web/gui/readme.md. (#10623, @OdysLam)
• Add resetting CapabilityBoundingSet workaround to the python.d collectors (that use sudo). (#10587, @ilyam8)
• Document the scheme option in the Elasticsearch collector. (#10572, @vjt)
• Update claiming instructions for Docker containers. (#10570, @Ferroin)

Bug fixes

• Fix the context filtering on the data query endpoint. (#10652, @stelfrag)
• Fix container/host detection in the system-info.sh script. (#10647, @ilyam8)
• Add a small delay to the ipv4_tcp_resets alarms. (#10644, @ilyam8)
• Fix collecting operstate for virtual network interfaces. (#10633, @ilyam8)
• Fix sendmail unrecognized option F error. (#10631, @ilyam8)
• Fix so that raw binary data should never be printed. (#10603, @rda0)
• Change KSM memory chart type to stacked. (#10598, @ilyam8)
• Allow the REMOVED alarm status via ACLK if the previous status was WARN/CRIT. (#10533, @stelfrag)
• Reduce excessive logging in the ACLK. (#10596, @underhood)

Release v1.29.1 is a hotfix release to address a crash in the Netdata Agent.

A locking bug in one of the internal collectors in Netdata could cause it to crash during shutdown in a way that would result in the Netdata Agent taking an excessively long time to exit.

Bug Fixes

• Fix crash during shutdown of cgroups internal plugin. (#10614, @mfundul)

Release v1.29.0

The v1.29.0 release of the Netdata Agent is a maintenance release that brings incremental but necessary improvements that make your monitoring experience more robust. We've pushed improvements and bug fixes to the installation and update scripts, enriched our library of collectors, and focused on fixing bugs reported by the community.

This release contains 2 new collectors, the migration of 3 collectors from Python to Go, 25 other improvements, 25 documentation updates, and 26 bug fixes.

At a glance

Netdata now collects and meaningfully organizes metrics from both the Couchbase JSON document database and the nginx-module-vts module for exposing metrics about NGINX virtual hosts. Click either of the links to head straight into the documentation that explains what they collect and how to configure both based on whether they're collecting over localhost or across nodes.

We've also migrated more collectors from Python to Go in our continued efforts to make data collection faster and more robust. The newest effort includes our Redis, Pika, and Energi Core Wallet collectors.

On the dashboard, we improved the responsiveness of panning forward and backward through historical metrics data by preventing unnecessary updates and reducing the number of calls. The charts should also now immediately update when you stop panning.

Acknowledgments

• @slavox for fixing temperature parsing in the hddtemp collector.
• @skibbipl for issuing a fix for the updater.
• @jsoref for the huge number of spelling fixes.
• @nabijaczleweli for the fix in the diskplace plugin.
• @Steve8291 for the documentation fix on using the Bash shell for debugging.
• @kdvlr for the added instructions on Telegram notifications.
• @grinapo for the improvement on the Python-based Fail2Ban collector.
• @dpsy4 for the support for per series styling for dygraphs
• @ski2per for adding nginxvts collector
• @hamedbrd for adding couchbase collector
• @g3offrey for improving prometheus collector default configuration

Improvements

• Reduce the number of alarm updates on ACLK. (#10524, @stelfrag)
• Remove unused entries from structures. (#10519, @stelfrag)
• Improve the retry/backoff during claiming. (#10482, @underhood)
• Support multiple chart label keys in data queries. (#10483, @stelfrag)
• Truncate excessive information from titles for apps and cgroups collectors. (#10479, @vlvkobal)
• Use mguid instead of hostname in the ACLK collector list. (#10394, @underhood)
• Cleanup and minor fixes to eBPF collector. (#10434, @thiagoftsm)
• Add _is_k8s_node label to the host labels. (#10501, @Ilyam8)
• Move ACLK into a legacy subfolder. (#10265, @underhood)
• Exclude autofs by default in the diskspace plugin. (#10441, @nabijaczleweli)
• Mark internal functions as static in health code. (#10518, @vkalintiris)
• Remove unused struct in health code. (#10517, @vkalintiris)
• Add support for per series styling for dygraphs. (#8668, @dpsy4)

Dashboard

• Fix minor vulnerability alert by updating socket-io dependency. (#10557, @jacekkolasa)
• Fix dygraph panning responsiveness, chart heights and performance improvements. (#10520, @jacekkolasa)
• Make legend position configurable. (#10565, @jacekkolasa)

Collectors

• Add Go version of the redis collector. (go.d.plugin#518, @Ilyam8)
• Add Go version of the pika collector. (go.d.plugin#518, @Ilyam8)
• Add Go version of the energis collector. (go.d.plugin#524, @thiagoftsm)
• Add a new nginxvts collector. (go.d.plugin#523, @ski2per)
• Add a new couchbase collector. (go.d.plugin#530, @hamedbrd)
• Add Traefik v2 to the prometheus collector default configuration. (go.d.plugin#539, @g3offrey)
• Add an expected_prefix configuration option to the prometheus collector. (go.d.plugin#541, @Ilyam8)
• Add patterns support to the filecheck collector. (go.d.plugin#538, @Ilyam8)

Packaging and installation

• Properly handle arguments and responses for triggering Docker builds. (#10545, @Ferroin)
• Properly handle saved temporary directory on updates. (#10550, @Ferroin)
• Update go.d.plugin version to v0.27.0. (#10544, @Ilyam8)
• Update messages about checksum validation failures on install. (#10448, @Ferroin)
• Switch to using system libwebsockets for RPM builds. (#10507, @Ferroin)
• Persist $TMPDIR from installer to updater. (#10384, @Ferroin)

Documentation

• Make some tweaks/improvements to configure docs. (#10528, @joelhans)
• Update Postgres collector doc to clarify how to install a required package. (#10532, @OdysLam)
• Add link to specific feedback megathread for the anomalies collector. (#10506, @andrewm4894)
• Update the instructions on how to install Netdata on pfSense. (#10466, @OdysLam)
• Fix documentation spelling mistakes (#10508, @jsoref)
• Add guide: Monitor and visualize anomalies with Netdata. (#10480, @joelhans)
• Add instructions on enabling explicitly disabled collectors (#10418, @joelhans)
• Mention PostgreSQL Prometheus Adapter in the documentation (#10487, @vlvkobal)
• Fix a typo in the python mysql documentation (#10467, @OdysLam)
• Fixes for SEO housekeeping/improvements (#10468, @joelhans)
• Add guide: Detect anomalies in nodes and applications with Netdata (#10451, @joelhans)
• Docs housekeeping for SEO and syntax, part 1 (#10388, @joelhans)
• Small updates, improvements, and housekeeping to docs (#10405, @joelhans)
• Change links at bottom of all install docs (#10416, @joelhans)
• Add missing section to Netdata style guide. (#10453, @joelhans)
• Update and improve the Netdata style guide. (#10433, @joelhans)
• Improve configuration docs with common changes and start/stop/restart directions. (#10415, @joelhans)
• Add instructions on which file to edit for Telegram. (#10398, @kdvlr)
• Add centralized Cloud notifications to core docs. (#10374, @joelhans)
• Fixes for SEO housekeeping/improvements. (#10468, @joelhans)
• Change links at bottom of all install docs. (#10416, @joelhans)
• GitHub action markdown link check update. (#10474, @jsoref)
• Change linting standard for Markdown lists. (#10371, @joelhans)
• Update main README with release news. (#10412, @joelhans)
• Improve the instructions on how to use the bash shell as user netdata for debugging. (#10425, @Steve8291)

Bug fixes

• Fix Docker image tagging for nightly builds. (#10584, @Ferroin)
• Fix container detection from systemd-detect-virt. (#10569, @cakrit)
• Fix Netdata Cloud support in RPM packages. (#10578, @Ferroin)
• Fix handling of TLS config so that cURL works in all cases. (#10491, @Ferroin)
• Fix function name in updater script. (#10462, @Ferroin)
• Fix handling of environment file in updater script. (#10447, @Ferroin)
• Fix bundling of libwebsockets in binary packages. (#10460, @Ferroin)
• Fix for the updater to use Python3 if Python is not available. (#10424, @skibbipl)
• Fix disconnect message sent via ACLK on agent shutdown (#10563, @underhood)
• Fix prometheus remote write header (#10560, @vlvkobal)
• Fix values in Prometheus export for metrics, collected by the Prometheus collector (#10551, @vlvkobal)
• Fix handling spaces in labels values in the prometheus collector (go.d.plugin#537, @Ilyam8)
• Fix mysql.slave_status alarm for go mysql collector (#10513, @Ilyam8)
• Make mdstat_mismatch_cnt alarm less strict (#10488, @Ilyam8)
• Dispatch cgroup discovery into another thread (#10399, @vlvkobal)
• Fix data source option for Prometheus web API in exporting configuration (#10397, @vlvkobal)
• Add Realtek network cards to the list of physical interfaces on FreeBSD (#10414, @vlvkobal)
• Fix anomalies collector custom model bug (#10459, @andrewm4894)
• Fix broken dbengine stress tests. (#10502, @mfundul)
• Fix segmentation fault in the agent (#10498, @mfundul)
• Fix memory allocation when computing standard deviation (#10484, @stelfrag)
• Fix temperature parsing in the hddtemp collector (#10429, @slavox)
• Fix postgres password bug and change default config (#10531, @OdysLam)
• Add handling "yes" and "no" and flexible space match in the python.d/fail2ban plugin (#10400, @grinapo)
• Fix for older compilers (#10470, @underhood)
• Fix spelling mistakes in the Python plugin and documentation. (#10525, @jsoref)

Release v1.28.0 is a hotfix release to address a deadlock in the Netdata Agent. We intended to release this hotfix as v1.27.1, but we can't backtrack on a release once we've begun to publish new Docker images and binary packages on other platforms.

If the Agent-Cloud link (ACLK) connection drops and the Agent fails to queue an on_connect message, it also fails to properly release a lock in the web server thread.

Bug fix

• Fix locking after on_connect failure. (#10401, @stelfrag)

Release v1.27.0

The v1.27.0 release of the Netdata Agent brings dramatic improvements to long-term metrics storage via the database engine, and new dashboard features like a time & date picker for visualizing precise timeframes. Two new collectors bring incredible new value to existing features, including a bit of machine learning magic.

This release contains 8 new collectors, 1 new notification method (2 others enhanced), 54 improvements, 41 documentation updates, and 58 bug fixes.

At a glance

The Netdata Agent now uses SQLite to store host, chart, and dimension metadata. This replaces the only metadata log files, which were located inside of the /var/cache/netdata/dbengine folder for both multihost and legacy children nodes streaming to a parent node. With SQLite powering the metadata log, you should notice faster Agent startups, as it no longer needs to replay metadata log files. The Agent no longer puts archived charts into memory on startup, further reducing memory usage. This is just the first of several improvements to the database engine and metadata log, with more coming in future releases.

The database engine now uses a new extent cache that improves query time by 10% under certain workloads and reduces disk I/O by 10%.

The Netdata Agent's local dashboard has received numerous improvements and bugfixes since v1.26. Perhaps most prominent is the new time & date picker, which helps you select precise timeframes when investigating an anomaly or troubleshooting an incident. See the dashboard repository's releases for the full changelog.

We also introduced two new collectors that monitor the Netdata Agent itself in unique ways. First is the anomalies collector, which uses machine learning (ML) to perform unsupervised anomaly detection on a node running the Netdata Agent. This collector trains itself to understand the baseline of specific charts, then charts anomalous data. A new alarms collector visualizes the volume of Netdata alarms triggered over time.

Acknowledgments

• Ali Dinifar, from ZDResearch, for reporting a stack buffer overflow vulnerability in web_client.c, which our team resolved within 24 hours.
• @ernestojpg for adding the number of allocated/stored objects within each storage to the varnish collector.
• @ernestojpg for adding support for MSE (Massive Storage Engine) to the varnish collector.
• @jurgenhaas for adding allocated space metrics to the oracledb collector.
• @autoalan for fixing a spelling mistake in the haproxy collector README.
• @ysamouhos for fixing a spelling mistake in UPDATE.md.
• @voriol for fixing the Ansible deployment guide.
• @scatenag for adding an option to exclude zero memory allocated users to the nvidia_smi collector.
• @fayak for adding per queue charts to the rabbitmq collector.
• @atnartur for fixing Markdown syntax in the custom dashboard documentation.
• @alexmyczko for removing redundant build dependencies from Debian control file.
• @KickerTom for fixing compilation with HTTPS disabled.
• @hexchain for fixing a database endless loop bug when cleaning obsolete charts.
• @wash2 for fixing the libreswan collector parsing.
• @Saruspete for fixing a platform dependent printf format.
• @KickerTom for fixing an eBPF cross compilation error and updating libnetdata headers to be compatible with C++.
• @WBTMagnum for fixing typos in the README.md.
• @Jiab77 for adding support to hide the SSO iframe.
• @martinpal for adding HBA drives support to the hpssa collector.
• @hamedbrd for fixing response and upstream response time histogram charts in the web_log collector.
• @hamedbrd for adding custom time fields feature to the web_log collector.
• @ski2per for adding directories size collection to the filecheck collector.

Improvements

• Add labels for Kubernetes pods and containers. (#10107, @ilyam8)
• Add plugin and module health entities. (#10041, @thiagoftsm)
• Migrate the metadata log to SQLite. (#10139, @stelfrag)
• Add an extent cache to the database engine. (#10293, @mfundul)
• Added new data query option allow_past. (#10112, @stelfrag)

Netdata Cloud

• Add the ability to query child nodes by their GUID. (#10030, @underhood)
• Add child availability messages to the ACLK. (#9918, @underhood)
• Add a metric showing how long a query spent in the queue. (#10016, @underhood)
• Completely hide the SSO iframe. (#10027, @Jiab77)

Collectors

• Add alarms obsoletion and disable alarms collector by default. (#10375, @ilyam8).
• Add calls for tcp_sendmsg, tcp_retransmit_skb, tcp_cleanup_rcv, udp_sendmsg, udp_recvmsg functions charts to the eBPF collector. (#10360, @thiagoftsm)
• Add two more insignificant warnings to suppress in anomalies collector. (#10369, @andrewm4894)
• Add the number of allocated/stored objects within each storage to the varnish collector. (#10329, @thiagoftsm)
• Add a wireless statistics collector. (#10052, @thiagoftsm)
• Add support for MSE (Massive Storage Engine) to the varnish collector. (#10317, @ernestojpg)
• Remove remove crit from unmatched alarms in the web_log collector. (#10280, @ilyam8)
• Add GPU key metrics (nvidia_smi collector) to dashboard_info.js. (#10230, @ilyam8)
• Add allocated space metrics to the oracledb collector. (#10197, @jurgenhaas)
• Restructure the eBPF collector to improve usability. (#10299, @thiagoftsm)
• Add an anomaly detection collector. (#10060, @andrewm4894)
• Add a Netdata alarms collector. (#10042, @andrewm4894)
• Add a configuration option to exclude users with zero memory allocated to the nvidia_smi collector. (#10098, @scatenag)
• Add per queue charts to the rabbitmq collector. (#10064, @fayak)
• Add support for HBA drives to the hpssa collector. (#10093, @martinpal)
• Update the cgroups collector default filtering by adding pod level cgroups. (#10095, @ilyam8)
• Add a Go version of the CouchDB collector (couchdb). (go.d.plugin/#453, @vlvkobal)
• Add collecting HTTP method per URL pattern (url_pattern option) to the web_log collector. (go.d.plugin/#458, @ilyam8)
• Add custom time fields feature to the web_log collector. (go.d.plugin/#467, @hamedbrd)
• Add a Go version of the PowerDNS Authoritative Nameserver collector (powerdns). (go.d.plugin/#501, @ilyam8)
• Add a Go version of the PowerDNS Recursor collector (powerdns_recursor). (go.d.plugin/#495, @ilyam8)
• Add a Go version of the PowerDNS DNSdist collector (dnsdist). (go.d.plugin/#504, @thiagoftsm)
• Add a Dnsmasq DNS Forwarder collector (dnsmasq). (go.d.plugin/#503, @ilyam8)
• Add collecting directories size to the filecheck collector. (go.d.plugin/#487, @ski2per)
• Add old systemd versions support to the systemdunits collector. (go.d.plugin/#502, @ilyam8)
• Add unmatched lines logging to the web_log collector. (go.d.plugin/#514, @ilyam8)

Notifications

• Add API V2 support to the PagerDuty health integration. (#10189, @thiagoftsm)
• Add threads support to the Google Hangouts health integration. (#10160, @thiagoftsm)
• Add a Opsgenie health integration. (#9879, @thiagoftsm)

Exporting

• Add HTTP and HTTPS support to the simple exporting connector. (#9911, @vlvkobal)

Packaging/installation

• Update React dashboard v2.11. (#10383, @jacekkolasa)
• Update go.d.plugin version to v0.26.2. (#10355, @ilyam8)
• Add numerous improvements to our Docker image. (#10338, @Ferroin)
• Use glibtoolize on macOS instead of regular libtoolize. (#10346, @Ferroin)
• Make the update script significantly more robust and user friendly. (#10261, @Ferroin)
• Update go.d.plugin version to v0.26.1. (#10319, @ilyam8)
• Update React dashboard v2.10.1. (#10314, @jacekkolasa)
• Update go.d.plugin version to v0.26.0 (#10284, @ilyam8)
• Update third-party static dependencies and use alpine 3.12. (#10241, @ktsaou)
• Update React dashboard to v2.9.2. (#10239, @jacekkolasa)
• Update eBPF collector to 0.4.9. (#10202, @thiagoftsm)
• Update go.d.plugin version to v0.25.0 (#10215, @ilyam8)
• Update React dashboard to v2.7.5. (#10179, @jacekkolasa)
• Add ability to use system libwebsockets instead of bundled version. (#9984, @underhood)
• Update the version of libJudy that we bundle to 1.0.5-netdata2. (#10158, @Ferroin)
• Update React dashboard to v2.7.4. (#10122, @jacekkolasa)
• Update go.d.plugin version to v0.24.0 (#10109, @ilyam8)
• Remove redundant build dependencies from Debian control file. (#10085, @alexmyczko)

CI/CD

• Switch to using official Docker actions for GHA CI. (#10364, @Ferroin)
• Explicitly set platform for Docker builds. (#10357, @Ferroin)
• Update distros for CI checks and package builds. (#10123, @Ferroin)
• Remove usage of deprecated GHA syntax. (#10154, @Ferroin)
• Split ReviewDog check to only run when relevant. (#10148, @Ferroin)

Documentation

• Add documentation for time & date picker in Agent and Cloud. (#10347, @joelhans)
• Add paragraph in anomalies collector README to ask for feedback. (#10363, @andrewm4894)
• Fix typo in performance guide. (#10386, @OdysLam)
• Update alarms collector README with fixed image.(#10348, @andrewm4894)
• Update macOS instructions with new Homebrew installation command. (#10379, @ktsaou)
• Update macOS instructions with cmake. (#10295, @joelhans)
• Add guide: Monitor any process in real-time with Netdata. (#10338, @joelhans)
• Improve core documentation to align with recent Netdata Cloud releases. (#10318, @joelhans)
• Add info about network usage requirements for the update script. (#10334, @Ferroin)
• Add new collectors to supported collectors list. (#10310, @joelhans)
• Document the Agent reinstallation process. (#10270, @joelhans)
• Add privacy information about ACLK connection. (#10292, @OdysLam)
• Improve python.d plugin PR checklist README section. (#10302, @andrewm4894)
• Fix a spelling error in the HAProxy documentation. (#10300, @autoalan)
• Fix a spelling error in the update documentation. (#10301, @ysamouhos)
• Add guide: How to optimize Netdata's performance. (#10271, @joelhans)
• Fix a syntax error in bug_report.md. (#10269, @OdysLam)
• Add new issue templates. (#10259, @OdysLam)
• Remove Docker example from update docs and add section to claim troubleshooting. (#10103, @joelhans)
• Improve docs to point users to proper configuration information. (#10254, @joelhans)
• Fix Docs GitHub Action with ignore list and update. (#10002, @joelhans)
• Fix broken links in documentation. (#10253, @joelhans)
• Fix a broken link in the Ansible guide. (#10232, @voriol)
• Add guide: Deploy Netdata with Ansible. (#10199, @joelhans)
• Fix a typo in the streaming doc. (#10225, @ilyam8)
• Fix repeated frontmatter in exporting docs. (#10211, @joelhans)
• Update k8s docs with new Helm repo. (#10172, @joelhans)
• Add a warning to exporting docs about an issue with the newest gRPC versions. (#10194, @vlvkobal)
• Add supported notification platforms to docs. (#10170, @joelhans)
• Add notices to FreeBSD/pfSense docs that they are community-supported. (#10171, @joelhans)
• Fix configuration category in the Prometheus remote write doc. (#10145, @OdysLam)
• Fix broken links. (#10115, @joelhans)
• Add documentation for Cloud Overview. (#10082, @joelhans)
• Update supported collectors list with new collectors. (#10102, @joelhans)
• Fix formatting source code blocks in custom dashboard page. (#10050, @atnartur)
• Add more robust documentation around updates. (#10100, @Ferroin)
• Update CONTRIBUTING.md with new meta title. (#10252, @joelhans)
• Update the Code of Conduct and widen scope to community. (#10186, @OdysLam)
• Update contact information in the Code of Conduct. (#10161, @aabatangle)
• Fix typos in the main README. (#10146, @WBTMagnum)
• Rewrite the repository's main README. (#10108, @joelhans)

Bug fixes

• Fix option parsing in kickstart.sh. (#10396, @Ferroin)
• Fix handling of dependencies on Gentoo. (#10382, @Ferroin)
• Fix crash in the eBPF plugin by initializing variables. (#10395, @thiagoftsm)
• Fix sending chart definition on every data collection in alarms collector. (#10378, @ilyam8)
• Fix a lock check. (#10385, @vlvkobal)
• Fix issue with chart metadata sent multiple times over ACLK. (#10381, @stelfrag)
• Fix a buffer overflow when extracting information from a streaming connection. (#10391, @stelfrag)
• Fix hostname configuration in the exporting engine. (#10361, @vlvkobal)
• Fix use of multiarch/qemu-user-static image for Docker builds. (#10352, @Ferroin)
• Fix handling of self-updating in updater script. (#10352, @Ferroin)
• Fix handling of Python dependency for RPM package. (#10345, @Ferroin)
• Fix handling of PowerTools repo on CentOS 8. (#10334, @Ferroin)
• Fix units and data source exporting options. (#10343, @vlvkobal)
• Fix building libwebsockets properly on macOS. (#10333, @Ferroin)
• Fix exporting config. (#10323, @vlvkobal)
• Fix health by disabling used_file_descriptors alarm. (#10328, @ilyam8)
• Fix GPU data filtering in the nvidia_smi collector. (#10312, @ilyam8)
• Fix username resolution in the nvidia_smi collector. (#10268, @ilyam8)
• Fix compilation with HTTPS disabled. (#10279, @KickerTom)
• Fix hostname when syslog is used in syslog health integration. (#10275, @thiagoftsm)
• Fix kernel crash caused by EBPF in Ubuntu 4.18.0-25 by adding it to the reject list. (#10262, @thiagoftsm)
• Fix streaming buffer size. (#10240, @vlvkobal)
• Fix database endless loop when cleaning obsolete charts. (#10236, @hexchain)
• Disable chart obsoletion code for archived chart creation. (#10231, @mfundul)
• Fix Prometheus remote write exporter so that it doesn't stop when data is not available for dimension formatting. (#10217, @vlvkobal)
• Fix memory calculation by moving shared from cached to used dimension. (#10183, @mfundul)
• Fix parsing in the libreswan collector. (#10190, @wash2)
• Fix an infinite loop in the statsd plugin (#10180, @vlvkobal)
• Fix two bugs related to version handling in install and update code. (#10162, @Ferroin)
• Fix builds using particular versions of Clang. (#10155, @Ferroin)
• Disregard host tags configuration pointer. (#10121, @mfundul)
• Fix platform dependent printf format. (#10120, @Saruspete)
• Fix compile error in CentOS 6. (#10110, @stelfrag)
• Fix cross compilation by properly disabling eBPF detection. (#10034, @KickerTom)
• Fix cgroups collector resolving container names in k8s. (#10072, @ilyam8)
• Fix a compilation warning. (#10320, @vlvkobal)
• Fix UUID_STR_LEN undefined on macOS. (#10313, @underhood)
• Fix python.d plugin runtime chart creation. (#10296, @ilyam8)
• Fix race condition in rrdset_first_entry_t() and rrdset_last_entry_t(). (#10276, @mfundul)
• Fix the data endpoint so that the context param is correctly applied to children. (#10290, @stelfrag)
• Fix Coverity errors (CID 364045,364046). (#10282, @stelfrag)
• Fix the elasticsearch_last_collected alarm. (#10226, @ilyam8)
• Fix spelling error in xenstat.plugin. (#10224, @ilyam8)
• Fix chart filtering. (#10218, @vlvkobal)
• Fix Coverity issues. (#10216, @vlvkobal)
• Fix libnetdata headers to be compatible with C++. (#10185, @KickerTom)
• Fix registry responses to remove caching. (#10181, @cakrit)
• Fix eBPF memory management. (#10096, @thiagoftsm)
• Fix overlapping memory issue. (#10097, @mfundul)
• Fix response and upstream response time histogram charts in the web_log collector. (go.d.plugin/#462, @hamedbrd)
• Fix logs timestamps always in UTC issue in the go.d.plugin (go.d.plugin#460, @ilyam8)
• Fix collecting slave status for MariaDB v10.2.0- in the mysql collector (go.d.plugin#465, @ilyam8)
• Fix cumulative_stats configuration option in the unbound collector (go.d.plugin#478, @ilyam8)
• Fix parsing configuration file (respect 'include-toplevel' directive) in unbound collector (go.d.plugin#480, @ilyam8)
• Fix handling charts with type.id >= 200 (netdata limit) in go.d.plugin (go.d.plugin#472, @ilyam8)
• Fix parsing version query response in the mysql collector (go.d.plugin#498, @ilyam8)
• Fix Netsplits chart dimensions algorithm in the the vernemq collector. (go.d.plugin#511, @ilyam8)
• Fix a typo in dashboard_info.js for VerneMQ. (#10223, @ilyam8)

Release v1.26.0

The v1.26.0 release of the Netdata Agent brings exciting new collectors written in Go, a new integration with the DevOps startup StackPulse, and massive improvements to the way users navigate Netdata's documentation. We've also added compatibility with an exciting new feature that's coming soon to Netdata Cloud—stay tuned!

This release contains 3 new collectors, 1 new notifcation method, 21 improvements, 13 documentation updates, and 12 bug fixes.

At a glance

The Netdata Agent can now collect metrics from files/directories, systemd units, and ISC DHCP servers. These new collectors are part of our larger effort to migrate all collectors to Go, which provides more extensibility compared to previous implementations. You can read about each of these new collectors in our docs: filecheck, systemd, isc_dhcpd.

We're excited to extend our health watchdog even further to integrate with StackPulse, which is designed to help SREs manage and respond to incidents with code and automation. You can read more about how to configure Netdata to send notifications to StackPulse in the docs.

We rearchitected our docs/education site, Netdata Learn, to focus on users actions rather than the Netdata Agent's hierarchy of code. The core docs now better guide users through the most important actions, such as configuring collectors and interacting with charts, independent of whether they use only the Netdata Agent, or the Agent in combination with Netdata Cloud. Of course, all of our reference documentation is still alive and kicking for those who want to dive into every configuration option or API query. We also revamped our guides page with better visuals, a search/filter, and more rational categories.

Acknowledgments

We're grateful to the Netdata community for their contributions for this release.

• @HolgerHees for fixing the comment syntax in Netdata's systemd file. (#10066,
• @Saruspete for fixing a file descriptor leak in the Infiniband colector (proc.plugin).
• @hamedbrd for adding a new Go-based systemd unit state collector and fixing gauges for the go.d.plugin/web_log collector.
• @chadknutson for adding chart for churn rates to python.d/rabbitmq.
• @hydrogen-mvm for adding a missing period in the Netdata dashboard.
• @roedie for adding a missing libelf-dev dependency.
• @Dim-P) and @disko for documentation improvements.

Improvements

• Add the ability to send Agent alarm notifications to StackPulse. (#9965, @thiagoftsm)
• Add a way to get build configuration info from the Agent. (#9913, @Ferroin)
• Add chart for churn rates to python.d/rabbitmq. (#10031, @chadknutson)
• Add failed dim to the connection_fails alarm in the Portcheck alarm. (#10048, @ilyam8)
• Improve the data query when using the context parameter (#9978, @stelfrag).
• Add a context parameter to the data endpoint. (#9931, @stelfrag)

Netdata Cloud

• Change default ACLK query thread count. (#10009, @underhood)
• Remove leading whitespace before JSON in ACLK. (#9998, @underhood)
• Allow using libwebsockets without SOCKS5. (#9973, @underhood)
• Add information about Cloud disabled status to -W buildinfo. (#9936, @underhood)

Collectors

• Update go.d.plugin version to v0.23.0. (#10046, @ilyam8)
  • Add new filecheck collector. (go.d.plugin/#445, @ilyam8)
  • Add new systemd unit state collector. (go.d.plugin/#439, @hamedbrd)
  • Add new ISC DHCP collector. (go.d.plugin/#451, @thiagoftsm)

Dashboard

• Add missing period in Netdata dashboard. (#9960, @hydrogen-mvm)
• Add missing tests to the web server. (#10008, @thiagoftsm)

Packaging/installation

• Rename NETDATA_PORT to NETDATA_LISTENER_PORT. (#10045, @knatsakis)
• Add a few changes that were missed by the systemd updater support. (#10007, @Ferroin)
• Switch to our installer's bundling code for libJudy in static installs. (#9988, @Ferroin)
• Add improved auto-update support. (#9966, @Ferroin)
• Add missing libelf-dev dependency. (#9974, @roedie)
• Update RPM spec file to use automatic dependency list generation. (#9937, @Ferroin)
• Add support for using /etc/cron.d for auto-updates. (#9598, @Ferroin)
• Add more stringent check for C99 support in configure script. (#9982, @Ferroin)

Documentation

• Add note about using nolock when debugging. (#10036, @andrewm4894)
• Update claiming document to instruct users to install uuidgen. (#9925, @OdysLam)
• Fix link in exporting document. (#10020, @Dim-P)
• Clean up and better cross-link new docsv2 documents. (#10015, @joelhans)
• Update FreeBSD documentation with updated packages. (#10005, @disko)
• Add documentation for claiming k8s parent pods and Prometheus service discovery. (#10001, @joelhans)
• Add docsv2 project to master branch. (#10000, @joelhans)
• Fix setting for disabling eBPF-apps.plugin integration. (#9967, @joelhans)
• Fix Stackpulse doc. (#9968, @thiagoftsm)
• Add persistent configuration details to Docker docs. (#9926, @joelhans)
• Add guide for monitoring Pi-hole and Raspberry Pi. (#9770, @joelhans)
• Add notice to Docker docs about systemd volumes. (#9927, @thiagoftsm)
• Add mirrored_hosts_status into Swagger docs. (#9867, @underhood)

Bug fixes

• Fix systemd comment syntax. (#10066, @HolgerHees)
• Fix file descriptor leak in Infiniband collector (proc.plugin). (#10013, @Saruspete)
• Fix the data endpoint to prioritize chart over context if both are present. (#10032, @stelfrag)
• Fix cleanup of obsolete charts. (#9985, @mfundul)
• Fix typos in installer functions. (#9992, @Ferroin)
• Fix typo inside netdata-installer.sh (#9962, @thiagoftsm)
• Fix build for the AWS Kinesis exporting connector. (#9823, @vlvkobal)
• Fix incorrect condition in updater type detection. (#10028, @Ferroin)
• Fix gauges for go.d.plugin/web_log collector. (#10029, @hamedbrd)
• Fix locking order to address CID_362348. (#9991, @stelfrag)
• Fix chart's last accessed time during context queries. (#9952, @stelfrag)
• Fix resource leak in case of malformed request to Netdata Cloud. (#9934, @underhood)

Release v1.25.0

The v1.25.0 release of the Netdata Agent is focused on improving Netdata's usability across the board. We added more customization to how the Prometheus collector implemented in v1.24 meaningfully visualizes metrics. In addition, we've focused on fixing bugs and ensuring that core functionality of the Netdata Agent, such as the ACLK, works more efficiently.

This release contains 1 new collector, 27 improvements, 15 documentation updates, and 59 bug fixes.

At a glance

Improved filtering and grouping for the Prometheus collector gives you more flexibility in how Netdata collects and visualizes metrics from more than 600 Prometheus endpoints. The Prometheus collector is designed to visualize every metric exposed on a Prometheus endpoint generically, but one chart for every metric is often not the most meaningful presentation. Filtering and grouping options bring the same "bespoke" feeling that you find in our other collectors, such as having input/output metrics on a single chart instead of two.

You can read about filtering and grouping in our documentation right now. If you haven't heard about the generic Prometheus collector, read our v1.24 blog post for details on why we continuously make Netdata more interoperable with other monitoring solutions.

We also made significant improvements to the robustness and responsiveness of the Agent-Cloud link (ACLK), which is used to stream metrics and alarm status if you sign up for Netdata Cloud and claim your nodes. The disconnect and reconnect process is now more reliable, and all metrics data is now Gzip compressed. Now that the payloads are smaller and more quickly processed, you'll see improved responsiveness when viewing dashboards in Netdata Cloud.

We added a new Elasticsearch collector, written in Go, to help you collect metrics from and monitor Elasticsearch instances. This collector is preinstalled with the Netdata Agent and often works with zero configuration, but can also be tweaked to collect only specific stats, gather metrics with TLS, and more. See the documentation for generated charts and configuration options.

Acknowledgments

We're grateful to the Netdata community for a huge wave of contributions for this release.

• @mklepaczewski for adding a JSON log parser to the go.d/web_log collector.
• @glesys-andreas for adding socket support for the go.d/phpfpm collector.
• @K900 for adding and documenting how to read container names from Podman.
• @pando85 for fixing the link to Caddyfile's basicauth in the Docker documentation.
• @roedie for improving Debian packaging by streamlining control and init files.
• @vsc55 for adding support for IP ranges in the python.d/isc_dhcpd collector.
• @mrbarletta for fixing MySQL collector documentation to mention the netdata user.
• @Saruspete for fixing RPM build script version issues.
• @michmach for improving the uninstall script to correctly state if the group was deleted.
• @Steve8291 for removing PrivateMounts in systemd journal logs.
• @mrbrutti for updating netdata-installer.sh to enable Netdata Cloud support in macOS.
• @weijing24 for adding RAM info for macOS to system-info.sh.
• @scottymuse for fixing latency-avg chart units in the python.d/dnsdist collector.
• @Ancairon for improving proc.plugin to collect the active processes limit on Linux systems.
• @scatenag for fixing TLS over LDAP in the python.d/openldap collector.
• @florianmagnin for adding new options to the python.d/varnish collector for multiple storage backends.
• @devinrsmith for fixing the print message when building for Ubuntu Focal.

Improvements

• Add code to release memory used by the global GUID map (#9729, @stelfrag)
• Add check for spurious wakeups (#9751, @vlvkobal)

Netdata Cloud

• Add v2 HTTP message with compression to ACLK (#9895, @underhood)
• Add version negotiation to ACLK (#9819, @underhood)
• Add claimed_id for child nodes streamed to their parents (#9804, @underhood)
• Update netdata-installer.sh to enable Netdata Cloud support in macOS (#9360, @mrbrutti)

Collectors

• Update go.d.plugin version to v0.22.0 (#9898, @ilyam8)
  • Add JSON parser to weblog collector (#417, @mklepaczewski)
• Update go.d.plugin version to v0.21.0 (#9881, @ilyam8)
  • Add new Elasticsearch collector (#421, @ilyam8)
  • Add filtering option to Prometheus collector (#416, @ilyam8)
  • Add custom grouping option to Prometheus collector (#418, @ilyam8)
  • Add add socket support to PHP-FPM collector (#402, @glesys-andreas)
• Add support for IP ranges to Python-based isc_dhcpd collector (#9755, @vsc55)
• Add Network viewer charts to ebpf.plugin (#9591, @thiagoftsm)
• Add collecting active processes limit on Linux systems (#9843, @Ancairon)
• Improve eBPF plugin by removing unnecessary debug messages (#9754, @thiagoftsm)
• Add CAP_SYS_CHROOT for netdata service to read LXD network interfaces (#9726, @vlvkobal)
• Add collecting maxmemory to python.d/redis (#9767, @ilyam8)
• Add option for multiple storage backends in python.d/varnish (#9668, @florianmagnin)

Dashboard

• Update dashboard v1.4.2 (#9837, @jacekkolasa)
  • Disable calls to netdata.cloud when --disable-cloud option is used during installation (#114, @jacekkolasa)
  • Fix Y-axis and auto-scaling for constant values (#115 & (#117, @jacekkolasa)
  • Fix broken dashboard when browser is configured to have no preferred language (#118, @jacekkolasa)
  • Fix d3-pie chart unit conversion on updates (#119, @jacekkolasa)
• Update dashboard to v1.3.1 (#9786, @jacekkolasa)
  • Fix stacked chart dimension visibility (#113, @jacekkolasa)

Packaging/installation

• Improve handling of offline installs (#9805, @Ferroin)
• Improve Debian packaging by streamlining control and init files (#8982, @roedie)
• Remove dependency on libJudy for systems which don't have it (#9859, @Ferroin)
• Add code to bundle libJudy on systems which do not provide a usable copy of it (#9776, @Ferroin)
• Improve temporary directory checking in installer and updater (#9797, @Ferroin)
• Add proper certificate handling cURL in our static build (#9733, @Ferroin)

Documentation

• Improve and correct vulnerability reporting instructions (#9696, @cakrit)
• Fix broken link in privacy policy (#9771, @joelhans)
• Update supported collectors doc to organize by type (#9513, @joelhans)
• Change instruction to reload HEALTH (#9869, @thiagoftsm)
• Fix typo in health documentation (#9860, @thiagoftsm)
• Fix broken Edit this page link in simple patterns doc (#9847, @joelhans)
• Remove Google Charts info from API doc (#9826, @joelhans)
• Fix broken link and clean up frontmatter in health docs (#9813, @joelhans)
• Improve dbengine docs and add new multihost setting (#9817, @joelhans)
• Improve health docs by adding daemon config to health section and standardizing IP references (#8837, @joelhans)
• Add and document support for reading container names from Podman in cgroups.plugin (#9474, @K900)
• Fix docker packaging caddyserver basicauth link (#9812, @pando85)
• Fix MySQL collector documentation to mention netdata user (#9555, @mrbarletta)
• Add community link to readme (#9602, @zack-shoylev)
• Add v1.24 news to main README (#9721, @aabatangle)

Bug fixes

• Fix setting the default value of the home directory to the environment's HOME (#9711, @cakrit)
• Fix memory mode none not dropping stale dimension data (#9917, @mfundul)
• Fix memory mode none not marking dimensions as obsolete (#9912, @mfundul)
• Fix race condition with orphan hosts (#9862, @mfundul)
• Fix the log level in cgroup-network helper (#9836, @vlvkobal)
• Fix empty dbengine files (#9820, @mfundul)
• Fix timestamps for global variables in Prometheus output (#9779, @vlvkobal)
• Fix long stats.d chart names (suggested by @vince-lessbits) (#9783, @amoss)
• Fix HTTP header for the remote write exporting connector (#9775, @vlvkobal)
• Fix netfilter to close when receiving a SIGPIPE (#9756, @thiagoftsm)
• Fix exporting update point (#9748, @vlvkobal)
• Fix for ignored LXC containers (#9645, @vlvkobal)
• Fix issue with missing alarms (#9712, @stelfrag)
• Fix child memory corruption by removing broken optimization in the sender thread (#9703, @amoss)
• Fix crash when receiving malformed labels via streaming. (#9715, @mfundul)
• Fix collectors on MacOS and FreeBSD to ignore archived charts. (#9695, @mfundul)
• Fix sending follow-up alarms when the initial status matches the notification (#9698, @cakrit)
• Fix typo in option name used to use bundled libJudy (#9893, @prologic)
• Fix handling of libJudy bundling for RPM packages (#9875, @Ferroin)
• Fix another typo in the libJudy bundling code (#9904, @Ferroin)
• Fix missing newline concatentation slash causing failures in RPM builds (#9900, @prologic)
• Fix high CPU in IPFS collector by disabling call to the /api/v0/stats/repo endpoint by default (#9687, @ilyam8)
• Fix flushing errors (#9738, @mfundul)
• Fix bugs in handling of Python 3 dependencies on install (#9839, @Ferroin)
• Fix RPM build script version issues (#9808, @Saruspete)
• Fix installation to not install eBPF plugin components when they shouldn't be installed (#9844, @vlvkobal)
• Fixed tmpdir handling failure on macOS/FreeBSD. (#9842, @Ferroin)
• Fix netdata-uninstaller.sh to correctly state whether the group was deleted (#9835, @michmach)
• Fix updater bug introduced by incomplete variable rename in #8808 (#9834, @Ferroin)
• Fixed bug in installer introduced by #8808 (#9831, @Ferroin)
• Fix systemd journal logs to remove PrivateMounts (#9619, @Steve8291)
• Fix netdata-updater.sh to correctly pass REINSTALL_OPTIONS (#8808, @prologic)
• Fix handling of offline installs (#9805, @Ferroin)
• Fix install if system does not have ebpf.plugin (#9809, @roedie)
• Fix packaging to enable eBPF collector only if enabled in config.h (#9752, @Saruspete)
• Fix numerous bugs in duplicate install handling (#9769, @Ferroin)
• Fix netdata/netdata Docker image size (#9669, @prologic)
• Fix global GUID map memory leak (#9725, @stelfrag)
• Fix buffer overflow in rrdr structure (#9903, @mfundul)
• Fix HTTP error messages in alarm notifications (#9887, @thiagoftsm)
• Fix lock order reversal (Coverity defect CID 361629) (#9888, @mfundul)
• Fix missing macOS RAM info in system-info.sh (#9882, @weijing24)
• Fix latency-avg chart units in python.d/dnsdist (#9871, @scottymuse)
• Fix TLS over LDAP in the python.d/openldap collector (#9853, @scatenag)
• Fix multi-host DB corruption when legacy metrics reside in localhost. (#9855, @mfundul)
• Fix compilation warnings on FreeBSD (#9845, @underhood)
• Fix proxy forwarding claim_id to old parent (#9828, @underhood)
• Fix old dashboard third-party packaging (#9814, @jacekkolasa)
• Fix loading custom dashboard_info in /old dashboard (#9792, @jacekkolasa)
• Fix unit tests for exporting engine (#9766, @vlvkobal)
• Fix code formatting for the mdstat collector (#9749, @vlvkobal)
• Fix health notifications configuration to clarify which notifications are received when the "|critical" limit is set (#9740, @cakrit)
• Fix print message when building for Ubuntu Focal (#9694, @devinrsmith)
• Fix alarm redirection link for Cloud to stop showing 404 (#9688, @cakrit)

Release v1.24.0

The v1.24.0 release of the Netdata Agent brings enhancements to the breadth of metrics we collect with a new generic Prometheus/OpenMetrics collector and enhanced storage and querying with a new multi-host database mode.

At a glance

This release broadens our commitment to open standards, interoperability, and extensibility with a new generic Prometheus collector that works seamlessly with any application that makes its metrics available in the Prometheus/OpenMetrics exposition format, including support for Windows 10 via windows_exporter. Netdata will autodetect over 600 Prometheus endpoints and instantly generate charts with all the exposed metrics, meaningfully visualized.

The Netdata Agent database engine enables long-term storage of per-second metrics inside the Agent using both RAM and disk space. In our new, multi-host database mode, parent and child nodes share resources in a single instance. Any pre-existing child node metrics remain in the legacy dbengine paths to ensure backward compatibility. To migrate those nodes to the new multi-host DB, simply delete those metric cache paths. This new mode supports distributed queries for the Agent as well as specific scenarios like streaming metrics from the child to parent database, streaming multiple child nodes to a single parent, and remembering which child or children are connected to the database even if streaming hasn't started.

Acknowledgments

• @lassebm for the FreeBSD interface error alarms
• @Saruspete for fixing the RPM default permissions for /usr/libexec/netdata
• @Steve8291 for adjusting check-kernel-config.sh to run in bash
• @bmatheny for adding pihole to the dns app group
• @tinyhammers for templatizing the health/megacli alarms

New Features

• Add generic Prometheus/OpenMetrics collector (#9644, @ilyam8)
• Add locking between different collectors for the same application, implemented in different technologies (#9584, @vlvkobal), (#9564, @ilyam8)
• Implement multihost database (#9556, @stelfrag)
• Add alarms for FreeBSD interface errors (#8340, @lassebm)

Documentation

• Add documentation to provide a comprehensive guide for package maintainers (#9467, @Ferroin)
• Add documentation to provide a comprehensive guide for package maintainers (#9467, @Ferroin)

Packaging / Installation

• Remove delay in updater script for non-interactive runs from install scripts. (#9589, @Ferroin)
• Remove runtime support for Polymorphic Linux from our Docker containers. (#9566, @Ferroin)
• Add better checks for existing installs to the kickstart scripts. (#9408, @Ferroin)
• Require cloud build to succeed in make dist checks. (#9218, @Ferroin)
• Use the libbpf library for the eBPF plugin (#9490, @vlvkobal)
• Fix Travis CI and remove deprecated/removed builds that have no upstream LXC image (#9630, @prologic)
• Fetch libbpf from netdata fork (#9637, @vlvkobal)
• Fix RPM default permissions for /usr/libexec/netdata (#9621, @Saruspete)
• Add eBPF collector support to DEB and RPM packages. (#9628, @Ferroin)
• Add sandboxing exception for /run/netdata. (#9613, @Ferroin)
• Remove delay in updater script for non-interactive runs from install scripts. (#9589, @Ferroin)
• Add proper handling for autogen on Ubuntu 18.04 (#9586, @Ferroin)
• Remove runtime support for Polymorphic Linux from our Docker containers. (#9566, @Ferroin)
• Add CAP_SYS_RESOURCE to capability bounding set. (#9569, @Ferroin)
• Add better checks for existing installs to the kickstart scripts. (#9408, @Ferroin)
• Enable simple sandboxing on systemd service (#9234, @Izorkin)
• Revert the eBPF package bundling that breaks the release and DEB packages. (#9552, @prologic)
• Add libbpf patch to make dist. (#9571, @Ferroin)

Bug Fixes

• charts.d: fix current_time_ms_from_date on macOS (#9636, @ilyam8)
• python.d/gearmand: handle func prefixes in status\n response (#9610, @ilyam8)
• Stop mdstat collector from looking up archived charts. (#9583, @mfundul)
• Fixes mempcpy->memcpy (#9575, @underhood)
• charts.d.plugin: never use -t option for timeout (#9568, @ilyam8)
• health/megacli: change all instances of alarm to template (#9553, @tinyhammers)
• Adjust check-kernel-config.sh to run in bash (#9633, @Steve8291)

Other Notable Changes

• Send netdata.public.unique.id (machine GUID) with claim (#9574, @underhood)
• Add pihole to the dns app group (#9557, @bmatheny)
• Implemented the HOST command in metadata log replay (#9489, @stelfrag)
• Implemented default disk space size calculation for multihost db (#9504, @stelfrag)
• Suppress warning -Wformat-truncation in ACLK (#9547, @underhood)
• Dashboard improvements (#9639, @jacekkolasa)

Netdata v1.23.2

Release v1.23.2 of the Netdata Agent is a patch for one significant issue.

PR #9491 fixed a buffer overrun vulnerability in Netdata's JSON parsing code. This vulnerability could be used to crash Agents remotely, and in some circumstances, could be used in an arbitrary code execution (ACE) exploit.

We strongly encourage all Netdata users to update their nodes to v1.23.2 as soon as possible.

This release also contains additional bug fixes and improvements.

Acknowledgements

• @Saruspete for adding Infiniband monitoring to Netdata!
• @meesaltena for fixing a typo in netdata-installer.sh.
• @anirudhdggl for tweaking the PyMySQL library to respect the my.cnf parameter when monitoring MySQL.
• @candrews for cleaning up the exporting engine by wrapping header definitions in compilation conditions.
• @RubenKelevra for deploying an update to the IPFS collector that makes it compatible with IPFS v0.5.0+.
• @vsc55 for adding support for returning headers using python.d's UrlService.

Improvements

• Add support for multiple ACLK query processing threads (#9355, @underhood)
• Add Infiniband monitoring to collector proc.plugin (#9091, @Saruspete)
• Change the HTTP method to make the IPFS collector compatible with 0.5.0+ (#9248, @RubenKelevra)
• Add support for returning headers using python.d's UrlService (#9236, @vsc55)

Documentation

• Fix broken link in Kavenegar notification doc (#9492, @joelhans)
• Add documentation for installing Netdata on k8s clusters (#9364, @joelhans)
• Add notices to packaging docs for access errors and Cloud dependencies (#9422, @joelhans)
• Fix broken link to Polyverse in Docker documentation (#9426, @joelhans)
• Add notice to eBPF documentation about incompatibility with static builds (#9418, @joelhans)

Packaging / installation

• Properly include eBPF collector in binary packages. (#9450, @Ferroin)
• Fix typo in netdata-installer.sh (#9433, @meesaltena)
• Fix broken link to Polyverse in Docker documentation (#9426, @joelhans)
• Add first class support for FreeBSD (#9413, @prologic)

CI/CD

• Disable CentOS 8.x CI (temporarily) (#9538, @prologic)
• Remove Fedora 30 from CI (#9274, @Ferroin)

Bug fixes

• Fix vulnerability in JSON parsing (#9491, @underhood)
• Fixed stored number accuracy (#9540, @stelfrag)
• Fix transition from archived to active charts not generating alarms (#9536, @mfundul)
• Fix PyMySQL library to respect my.cnf parameter (#9526, @anirudhdggl)
• Remove health from archived metrics (#9520, @mfundul)
• Update exporting engine to read the prefix option from instance config sections (#9463, @vlvkobal)
• Fix display error in Swagger API documentation (#9417, @underhood)
• Wrap exporting engine header definitions in compilation conditions (#9458, @candrews)
• Improve cgroups collector to autodetect unified cgroups (#9249, @underhood)
• Fix CMake build failing if ACLK is disabled (#9537, @underhood)
• Fix now_ms in charts.d collector to prevent tc-qos-helper crashes (#9510, @ilyam8)
• Fix python.d crashes by adding a lock to stdout write function (#9508, @ilyam8)
• Fix an issue with random crashes when updating a chart's metadata on the fly (#9509, @stelfrag)
• Fix ACLK protocol version always parsed as 0 (#9502, @underhood)
• Fix the check condition for chart name change (#9503, @stelfrag)
• Fix the exporting engine unit tests (#9460, @vlvkobal)
• Fix a Coverity defect for resource leaks (#9462, @vlvkobal)

Netdata v1.23.1

Release v1.23.1 of the Netdata Agent is a patch for two significant issues.

PR #9436 fixed an issue where dimensions were marked obsolete and archived simultaneously, which caused segmentation faults. We're grateful to marioem, who first reported the issue, and other members of the Netdata community who contributed their insights and valuable log information, which we used to diagnose and fix the bug.

PR [#9428] fixed a significant issue with duplicate alarm IDs, which caused issues in how alarms were sent and displayed in Netdata Cloud.

This release also contains a few additional bug fixes that were not fully reviewed before the release of v1.23.0.

Bug fixes

• Disallow dimensions and chart being obsolete and archived simultaneously. (#9436, @mfundul)
• Fix duplicate alarm ids in health-log.db (#9428, @stelfrag)
• Show cgroups/containers ran by Kubelet without access to Kubernetes cluster information (#9321, @cakrit)
• Fix children version on stream (#9438, @thiagoftsm)
• Fix internal registry (#9434, @thiagoftsm)
• Correct virtualization detection in system-info.sh (#9425, @Ferroin)
• Fix the unittest execution (#9445, @thiagoftsm)
• Update description in registry with minor copy edits (#9441, @amoss)
• Stop reading from /proc/sys/kernel/osrelease at trailing newline (#9374, @sjuxax)

Release v1.23.0

The v1.23.0 release of the Netdata Agent is all about unlocking new depths of visibility for your applications, services, and systems. We have Kubernetes service discovery, new eBPF metrics like virtual filesystem switch and bandwidth per process out of the Linux kernel at event frequency, more interoperability with your monitoring stack thanks to a new exporting engine, and much more.

This release contains 2 new collectors, 1 new exporting connector, 1 new alarm notification method, 55 improvements, 45 documentation updates, and 40 bug fixes.

At a glance

Our service discovery collector detects Kubernetes (k8s) pods and immediately collects metrics from 22 different services as the associated pods are created, destroyed, and scaled. Service discovery is installed when you use our Helm chart, which means you can now collect and visualize service-, pod-, Kubelet-, kube-proxy-, and node-level k8s metrics with one helm install command and zero configuration. All our Kubernetes monitoring components are open source and free for clusters of any size.

Our low-level Linux kernel monitoring via eBPF is now supercharged. Thanks to an integration with apps.plugin, you can now monitor how a specific application interacts with the Linux kernel. This update also includes new metrics, such as virtual filesystem switch, bandwidth per process, and much more. Netdata collects these metrics at an event frequency, even better than our famous 1s granularity, so that you can debug applications or anomalies with pinpoint accuracy. The eBPF collector is also now installed and enabled by default except on static builds.

Read our guide on troubleshooting apps with eBPF metrics for more details.

Netdata is now more interoperable with your existing monitoring stack thanks to the exporting engine, which replaces the backends system. You can now export to multiple external databases through Graphite, Google Cloud Pub/Sub, Prometheus remote write, MongoDB, and JSON connectors, plus others. Send metrics as soon as they're collected to enrich single pane of glass views or analyze Netdata's metrics with machine learning.

Read our guide on exporting metrics to Graphite for specifics on just one of many pipelines you can set up to archive your Netdata metrics.

We're also releasing an improvement for the availability of your monitoring and metrics: persistent metadata. The Agent now writes metadata to disk alongside metrics to allow access to non-active charts from Netdata Cloud and enable future features.

We added some enhancements to our documentation site, including a new guides section. We'll continue to populate with more use case- and scenario-based content to help you monitor, troubleshoot, visualize, and export your Netdata metrics.

Acknowledgments

• okias for adding support for Matrix notifications.
• elelayan for adding an OSD size collection chart to the Ceph collector.
• vsc55 for fixing the required packages for Gentoo builds.
• rushikeshjadhav for fixing the Xenstat collector to correctly track the last number of vCPUs.
• Saruspete for removing conflicting EPEL packages.
• MrFreezeex for fixing suid bits in Debian packaging.
• Neamar for fixing a typo in the dashboard's description of the mem.kernel chart.
• jeffgdotorg for fixing incorrectly formatted TYPE lines in the Prometheus backend/exporter.
• tnyeanderson for continuing to improve his dash.html custom dashboard.
• dpsy4 for fixing our Swagger API file.
• araemo for fixing alarms around RAM usage in ZFS systems.
• slavaGanzin for implementing a fix to the PostgreSQL collector.
• pkrasam, thoggs, oneoneonepig, Steve8291, stephenrauch, waybeforenow, zvarnes, electropup42, cherouvim, thenktor, webash and gruentee for contributing documentation changes.

Improvements

• Added libuv thread names support to FATAL log level. (#9382) by mfundul
• Updated the React dashboard to v1.0.14_2. (#9350) by jacekkolasa
• Improved PR guidelines for developers and contributors. (#8809) by prologic
• Removed master-slave verbiage and replaced it with parent-child. (#9323) by amoss, (#9312) by joelhans
• Added support for persistent metadata. (#9324) by stelfrag
• Add verbose prints when spawn server fails to spawn. (#9305) by mfundul
• Updated streaming protocol calculate clock-slew and gap-size when child nodes reconnect to a parent. (#9214) by amoss
• Implemented a new incremental parser for internal plugins and child nodes. (#9074) by stelfrag
• Improved database engine by reducing its minimum size to 64 MiB. (#9094) by mfundul
• Added alphabetical sort and automatic scroll to dash.html. (#8762) by tnyeanderson
• Added a spawn server to improved Agent scalability by reducing the impact of alarm execution and notification to critical sections in the main health thread. (#8407) by mfundul

Netdata Cloud

• Added metrics for ACLK performance and status to the Netdata Monitoring section of the dashboard. (#9269) by underhood
• Improved the node re-claiming process by regenerating the topic base. (#9044) by amoss

Collectors

• Updated the Go orchestrator to v0.19.2. (#9340) by ilyam8
• Added the agent-service-discovery collector plugin to apps_group.conf. (#9315) by ilyam8
• Improved consistency of Kubernetes cgroup names. (#9303) by cakrit
• Updated the Go orchestrator to v0.19.1. (#9309) by ilyam8
• Added imunify and lsphp to apps_groups.conf. (#9284) by thiagoftsm
• Updated the Go orchestrator to v0.19.0. (#9294) by ilyam8
• Added support for the eBPF collector in static installations (kickstart-static64.sh). (#8879) by prologic
• Updated the eBPF kernel-collector to v0.4.0. See the changelog for details. (#9212) by Ferroin
• Added integration between ebpf.plugin and apps.plugin. (#9178) by thiagoftsm
• Converted the eBPF collector into a modular design to allow multiple eBPF programs to run in parallel. (#9148) by thiagoftsm
• Added an OSD size collection chart to the Ceph collector. (#8649) by elelayan
• Updated the eBPF kernel-collector to v0.2.0. See the changelog for details. (#9118) by prologic
• Improved system-info.sh to better handle certain cases when gathering info on the system's disk capacity. (#7902) by Ferroin
• Changed the eBPF collector to install and enable it by default. (#8665) by Ferroin
• Enhanced the Samba collector to only use sudo when not running as the root user. (#9038) by Duffyx
• Renamed the eBPF collector from ebpf_process.plugin to ebpf.plugin. (#8822) by thiagoftsm
• Added more command line options to the eBPF collector to support upcoming features. (#8879) by thiagoftsm
• Added compatibility for Varnish Cache Plus in the varnish collector. (#8940) by pgjavier

Packaging/installation

• Added new streaming files into CMake build. (#9316) by underhood
• Added support for macOS/Homebrew in install-required-packages.sh. (#8286) by Ferroin
• Improved reliability of checksums for kickstart.sh/kickstart-static64.sh installation scripts. (#9165) by prologic
• Added required bundle for libuuid on ClearLinux. (#9060) by Ferroin
• Removed conflicting EPEL packages. (#9108) by Saruspete

Exporting

• Moved nc backend to exporting. (#9030) by thiagoftsm
• Added missing checks to exporting engine. (#9034) by thiagoftsm
• Added new alarms for exporting engine resource usage and deprecation of backends. (#9075) by thiagoftsm
• Added an error report to the AWS Kinesis connector. (#9048) by thiagoftsm
• Added memory cleanup to remaining exporting connectors. (#9098) by thiagoftsm
• Added a warning if the exporting engine's update interval is not a multiple of the database's update interval. (#9131) by vlvkobal
• Added anonymous statistics to exporting engine to collect usage data. (#9125) by vlvkobal
• Improved dynamic memory cleanup for Pub/Sub exporting connector. (#9112) by vlvkobal
• Improved dynamic memory cleanup for the MongoDB exporting connector. (#9103) by vlvkobal
• Finalized the main cleanup function for the exporting engine. (#9099) by vlvkobal
• Added a function to help clean up memory on exit. (#9081) by vlvkobal
• Added a Google Cloud Pub/Sub connector to the exporting engine. (#8855) by vlvkobal

Notifications

• Added support for Matrix notifications. (#9196) by okias

CI/CD

• Removed Gentoo from CI checks. (#9327) by prologic
• Added a random offset to the update script when running non-interactively. (#9245) by Ferroin
• Added a CI check for building against LibreSSL. (#9216) by prologic
• Added a health check functionality to Docker images. (#9172) by Ferroin
• Added CI for static builds of the Netdata Agent (used by kickstart-static64.sh). (#9130) by prologic
• Removed deprecated documentation Dockerfile and associated Docker Hub image. (#9126) by prologic
• Removed deprecated documentation tooling. (#8783) by prologic
• Added a CI job to check Markdown links during PRs. (#9003) by joelhans
• Removed Polyverse Polymorphic Linux from Docker builds to reduce the image size. (#8802) by Ferroin

Documentation

• Fixed a typo in the Synology installation documentation. (#9400) by pkrasam
• Added a guide for troubleshooting with eBPF metrics. (#9352) by joelhans
• Improved the FreeBSD installation documentation. (#9116) by thoggs
• Added a missing slash to the claiming documentation. (#9257) by oneoneonepig
• Changed the recommended repository for CentOS 8 users. (#9308) by Ferroin
• Added a guide for exporting metrics to Graphite. (#9285) by joelhans
• Added a link in the eBPF documentation to the kernel documentation for ftrace. (#9211) by Steve8291
• Fixed curly to straight apostrophe. (#8723) by zack-shoylev
• Added documentation and dashboard information for new eBPF-apps.plugin integration. (#9199) by thiagoftsm
• Moved and refactored docs to accomodate new Guides section on Learn. (#9266) by joelhans
• Removed outdated information/links from main README and registry doc. (#9265) by joelhans
• Added notes/known issues section to installation page. (#9053) by joelhans
• Fixed ambiguity in health reference for of and foreach options in lookup line. (#9255) by underhood
• Added a new "home base" document for the exporting engine. (#9246) by joelhans
• Improved database engine documentation for streaming setups. (#9177) by joelhans
• Fixed typo in eBPF collector README.md. (#9205) by Steve8291
• Fixed typo in README.md. (#9151) by stephenrauch
• Removed the "experimental" label from the exporting engine documentation. (#9171) by vlvkobal
• Fixed typo in step 3 of step-by-step guide. (#9150) by waybeforenow
• Added a Certbot troubleshooting section to step 10 of the step-by-step guide. (#9000) by Jelmerrevers
• Updated eBPF documentation to reflect default enabled status. (#9105) by joelhans
• Added ACLK connection details. (#9047) by zack-shoylev
• Added CMake to the list of packages to install on FreeBSD installations. (#9031) by zvarnes
• Improved Synology installation document with better formatting and instructions. (#8658) by thenktor
• Updated pfSense installation document with new packages and processes. (#8544) by electropup42
• Updated documentation contributing guidelines and Netdata style guide. (#8781) by joelhans
• Added links to promote database engine calculator. (#9067) by joelhans
• Updated exporting engine documentation to prepare for enabling it by default. (#9066) by vlvkobal
• Added requirements to the ProxySQL collector documentation. (#9071) by ilyam8
• Added proc.plugin configuration example for high-processor systems. (#9062) by joelhans
• Added frontmatter for exporting connectors. (#9052) by joelhans
• Fixed grammar error in HAProxy documentation. (#8703) by cherouvim
• Updated FreeBSD package installation documentation. (#8643) by thenktor
• Fixed docker run instruction in claiming document. (#9058) by ilyam8
• Added a note about restarting a node during reclaiming. (#9049) by zack-shoylev
• Removed mentions of old Cloud and replaced them with new Cloud/dashboard. (#8874) by joelhans
• Fixed broken link in web server log guide on GitHub. (#9033) by joelhans
• Removed emoji from step-by-step guide. (#8872) by MeganBishopMoore
• Added text to claiming documentation about reclaiming. (#9027) by joelhans
• Updated daemon output with new URLs and dates. (#8965) by joelhans
• Added netdatalib and netdatacache volumes to the Docker-with-Caddy documentation. (#8999) by webash
• Fixed an incorrect file name in the Go-based web log collector. (#8964) by gruentee
• Removed incorrect UNUSED from flood protection configuration options documentation. (#8964) by mfundul
• Fixed internal links and removed obsolete admonitions. (#8946) by joelhans
• Updated docs with go-live claiming and ACLK information. (#8960) by joelhans

Bug fixes

• Fixed a Coverity defect. (#9402) by amoss
• Fix a bug in the simple exporting connector that caused crashes when both opentsdb:https and another connector were enabled together. (#9389) by vlvkobal
• Fixed missing host variables on stream. (#9396) by thiagoftsm
• Fixed race-hazard in streaming during the shutdown sequence. (#9370) by amoss
• Fixed error handling and recovery during compaction and metadata log replay. (#9354) by stelfrag
• Fixed ACLK shutdown sequence. (#9367) by underhood
• Fixed logging by replacing assert() calls with new fatal_assert(). (#9349) by mfundul
• Fixed issues with CentOS 6 installations by getting Netdata execution path early to avoid user permission issues. (#9339) by mfundul
• Fixed issues with ebpf.plugin and apps.plugin integration. (#9333) by thiagoftsm
• Fixed Coverity warnings in database. (#9338) by mfundul
• Fixed compiler warnings from the database when the Agent is compiled with the --disable-cloud flag. (#9337) by stelfrag
• Fixed invalid memory access in databases to avoid Coverity errors. (#9326) by stelfrag
• Fixed broken updates to do enabling the eBPF collector by default with a dummy --enable-ebpf flag. (#9310) by Ferroin
• Fixed exporting to Cortex by adding an additional HTTP header to the Prometheus remore write connector. (#9302) by vlvkobal
• Fixed a race hazard causing crashes in streaming configurations. (#9297) by amoss
• Fixed handling of OpenSSL on CentOS/RHEL by bundling a static copy and selecting a configuration directory at install time. (#9263) by Ferroin
• Fixed static installation from overwriting netdata.conf. (#9174) by Ferroin
• Fixed compilation on older systems (Ubuntu 14.04 LTS, Debian 8, CentOS 6). (#9198) by ktsaou
• Fixed broken unit tests for the exporting engine. (#9183) by vlvkobal
• Fixed an issue with the exporting engine not cleaning a string on exit. (#9188) by vlvkobal
• Fixed issue with incremental parser breaking CMake builds. (#9186) by stelfrag
• Fixed the eBPF collector failing to install on certain systems. (#9182) by prologic
• Fixed Coverity warning. (#9180) by thiagoftsm
• Fixed required packages for Gentoo builds. (#9141) by vsc55
• Fixed Coverity warning. (#9157) by stelfrag
• Fixed broken collector plugins due to bug in parser. (#9158) by stelfrag
• Fixed the Xenstat collector to correctly track the last number of vCPUs. (#8720) by rushikeshjadhav
• Fixed incorrect link in install-required-packages.sh to help users submit a GitHub issue. (#8911) by prologic
• Fixed enable/start of netdata service in Debian package. (#9005) by MrFreezeex
• Fixed buffer splitting in the Kinesis exporting connector. (#9122) by vlvkobal
• Fixed suid bits on plugin for Debian packaging. (#8996) by MrFreezeex
• Fixed zombie procesess in Docker image by restoring SIGCHLD signal handler. (#9107) by mfundul
• Fixed static installation to not overwrite netdata.conf when updating. (#9046) by Ferroin
• Fixed typo in the dashboard's description of the mem.kernel chart. (#9096) by Neamar
• Fixed incorrectly formatted TYPE lines in the Prometheus backend/exporter. (#9086) by jeffgdotorg
• Fixed error handling in the exporting connector. (#8910) by vlvkobal
• Added a missing bracket to the Netdata API swagger .json file. (#8814) by dpsy4
• Fixed the health entity calculation used for ram_in_use and used_ram_to_ignore in systems using ZFS. (#8913) by araemo
• Fixed incorrect hostnames in the exporting engine. (#8892) by vlvkobal
• Fixed an issue with the PostgreSQL collector to correctly ignore template1/template0 databases. (#8929) by slavaGanzin

Netdata v1.22.1

Release v1.22.1 is a hotfix release to address issues related to packaging and how Agents connect to Netdata Cloud.

With packaging, we fixed an error that caused DEB and RPM packages to only display the old dashboard and not the new React version. We also fixed an issue that caused Netdata Docker containers to fail due to incorrect permissions. Finally, we ensured JSON-C is correctly fetched and built for compatibility with Netdata Cloud.

We appreciate our community's help in identifying and diagnosing these issues so we could fix them quickly.

For Netdata Cloud, we optimized the on-connect payload sent through the Agent-Cloud link to improve latency between Agents and Cloud. We also removed a check for old alarm status when sending alarms to Cloud via the ACLK.

Finally, we made a fix that ensures Agents running on systems using the musl C library can receive auto-updates.

Bug fixes

• Fixed the latency issue on the ACLK and suppress the diagnostics. (#8992) by amoss and stelfrag
• Restored old semantics of "netdata -W set" command. (#8987) by mfundul
• Added JSON-C packaging fils to make dist. (#8986) by Ferroin
• Fixed bundling of React dashboard in DEB and RPM packages. (#8988) by Ferroin
• Removed check for old alarm status. (#8978) by stelfrag
• Fixed shutdown via netdatacli with musl C library. (#8931) by mfundul

Release v1.22.0

Release v1.22.0 marks the official launch of our rearchitected Netdata Cloud! This Agent release contains both backend and interface changes necessary to connect your distributed nodes to this dramatically improved experience.

Netdata Cloud builds on top of our open source monitoring Agent to give you real-time visibility for your entire infrastructure. Once you've connected your Agents to Cloud, you can view key metrics, insightful charts, and active alarms from all your nodes in a single web interface. When an anomaly strikes, seamlessly navigate to any node to troubleshoot and discover the root cause with the familiar Netdata dashboard.

Sign in to Cloud and read our Get started with Cloud guide for details on updating your nodes, claiming them, and navigating the new Cloud.

While Netdata Cloud offers a centralized method of monitoring your Agents, your metrics data is not stored or centralized in any way. Metrics data remains with your nodes and is only streamed to your browser through Cloud.

In addition, Cloud only expands on the functionality of the wildly popular free and open source Agent. We will never make any of our open source Agent features Cloud-exclusive, and we will actively continue to develop the Agent so that we can integrate new features with Netdata Cloud.

This release also contains 1 new collector, 1 new exporting connector, 1 new alarm notification method, 27 improvements, 16 documentation updates, and 22 bug fixes.

At a glance

We added a new collector called whoisquery that helps you monitor a domain name's expiration date. You can track as many domains as you'd like, and set custom warning and critical thresholds for each. For more information on setup and configuration, see the Whois domain expiry monitoring documentation.

We added a new connector to our experimental exporting engine: Prometheus remote write. You can use this connector to send Netdata metrics to your choice of more than 20 external storage providers for long-term archiving and further analysis.

Our new documentation experience is now available at Netdata Learn! We encourage you to try it out and give us feedback or ask questions in our GitHub issues. Learn features documentation for both the Agent and Cloud in separate-but-connected vaults, which streamlines the experience of learning about both products.

While Learn only features documentation for now, we plan on releasing more types of educational content serving the Agent's open-source community of developers, sysadmins, and DevOps folks. We'll have more to announce soon, but in the meantime, we hope you enjoy what we believe is a smoother (and prettier) docs experience.

Acknowledgments

• amishmm for updating netdata.conf and netdata.service.v235.in.
• adamwolf for fixing a typo in netdata-installer.sh.
• lassebm for fixing a crash when shutting down an Agent with the ACLK disabled.
• yasharne for adding a new whoisquery collector and for adding health alarm templates for both the whoisquery and x509check collectors.
• illumine for adding Dynatrace as a new alarm notification method.
• slavaGanzin, carehart, Jiab77, and IceCodeNew for documentation fixes and improvements.

Breaking changes

• The previous iteration of Netdata Cloud, accessible through various Sign in and Nodes view (beta) buttons on the Agent dashboard, is deprecated in favor of the new Cloud experience.
• Our old documentation site (docs.netdata.cloud) was replaced with Netdata Learn. All existing backlinks redirect to the new site.
• Our localization project is no longer actively maintained. We're grateful for the hard work of its contributors.

Improvements

Netdata Cloud

• Enabled support for Netdata Cloud. (#8478), (#8836), (#8843), (#8838), (#8840), (#8850), (#8853), (#8866), (#8871), (#8858), (#8870), (#8904), (#8895), (#8927), (#8944) by amoss, jacekkolasa, Ferroin, prologic, mfundul, underhood, and stelfrag.
• Added TTL headers to ACLK responses. (#8760) by amoss
• Improved the thread exit fixes in #8750. (#8750) by amoss
• Added support for building libmosquitto on FreeBSD/macOS. (#8254) by Ferroin
• Improved ACLK reconnection sequence. (#8729) by stelfrag
• Improved ACLK memory management and shutdown sequence. (#8611) by stelfrag
• Added session-id to ACLK using connect timestamp. (#8633) by amoss

Collectors

• Improved the index size for the eBPF collector. (#8743) by thiagoftsm
• Added health alarm templates for the whoisquery collector. (#8700) by yasharne
• Added a whoisquery collector. go.d.plugin/#368 by yasharne
• Removed an automatic restart of apps.plugin. (#8592) by vlvkobal

Packaging/installation

• Added missing NETDATA_STOP_CMD in netdata-installer.sh. (#8897) by prologic
• Added JSON-C dependency handling to installation and packaging. (#8776) by Ferroin
• Added a check to wait for a recently-published tag to appear in Docker Hub before publishing new images. (#8713) by knatsakis
• Removed obsolete scripts from Docker images. (#8704) by knatsakis
• Removed obsolete DEVEL support from Docker images. (#8702) by knatsakis
• Improved how we publish Docker images by pushing synchronously. (#8701) by knatsakis

Exporting

• Enabled internal statistics for the exporting engine in the Agent dashboard. (#8635) by vlvkobal
• Implemented a Prometheus exporter web API endpoint. (#8540) by vlvkobal

Notifications

• Added a certificate revocation alarm for the x509check collector. (#8684) by yasharne
• Added the ability to send Agent alarm notifications to Dynatrace. (#8476) by illumine

CI/CD

• Disabled document-start yamllint check. (#8522) by ilyam8
• Simplified Docker build/publish scripts to support only a single architecture. (#8747) by knatsakis
• Added Fedora 32 to build checks. (#8417) by Ferroin
• Added libffi to ArchLinux CI tests as a workaround for an upstream bug. (#8476) by Ferroin

Other

• Updated main copyright and links for the year 2020 in daemon help output. (#8937) by zack-shoylev
• Moved bind to to [web] section and update netdata.service.v235.in to sync it with recent changes. (#8454) by amishmm
• Put old dashboard behind a prefix instead of using a script to switch. (#8754) by Ferroin
• Enabled the truthy rule in yamllint. (#8698) by ilyam8
• Added Borg backup, Squeezebox servers, Hiawatha web server, and Microsoft SQL to apps.plugin so that it can appropriately group them by type of service. (#8646), (#8655), (#8656), and (#8659) by vlvkobal

Documentation

• Add custom label to collectors frontmatter to fix sidebar titles in generated docs site at learn.netdata.cloud. (#8936) by joelhans
• Added instructions to persist metrics and restart policy in Docker installations. (#8813) by joelhans
• Fixed modifier in Nginx guide to ensure correct paths and filenames. (#8880) by slavaGanzin
• Added documentation for working around Clang build errors. (#8867) by Ferroin
• Fixed typo in Docker installation instructions. (#8861) by carehart
• Added Docker instructions to claiming docs. (#8755) by joelhans
• Capitalized title in streaming doc. (#8712) by zack-shoylev
• Updated pfSense doc and added warning for apcupsd users. (#8686) by cryptoluks
• Improved offline installation instructions to point to correct installation scripts and clarify process. (#8680) by IceCodeNew
• Added missing path to the process of editing charts.d.conf. (#8740) by Jiab77
• Added combined claiming and ACLK documentation. (#8724) by joelhans
• Standardized how we link between various Agent-specific documentation. (#8638) by joelhans
• Pinned mkdocs-material to re-enable Netlify builds of documentation site. (#8639) by joelhans
• Updated main README.md with v1.21 release news. (#8619) by joelhans
• Changed references of MacOS to macOS. (#8562) by joelhans

Bug fixes

• Fixed kickstart error by removing old cron symlink. (#8849) by prologic
• Fixed bundling of old dashboard in binary packages. (#8844) by Ferroin
• Fixed typo in netdata-installer.sh. (#8811) by adamwolf
• Fixed failure output during installations by removing old function call. (#8824) by Ferroin
• Fixed bundle-dashboard.sh script to prevent broken package builds. (#8823) by prologic
• Fixed mdstat failed devices alarm. (#8752) by ilyam8
• Fixed rare race condition in old Cloud iframe. (#8786) by jacekkolasa
• Removed no-clear-notification options from portcheck health templates. (#8748) by ilyam8
• Fixed issue in system-info.shregarding the parsing of lscpu output. (#8754) by Ferroin
• Fixed old URLs to silence Netlify's mixed content warnings. (#8759) by knatsakis
• Fixed master streaming fatal exits. (#8780) by thiagoftsm
• Fixed email authentiation to Cloud/Nodes View. (#8757) by jacekkolasa
• Fixed non-escaped characters in private registry URLs. (#8757) by jacekkolasa
• Fixed crash when shutting down an Agent with the ACLK disabled. (#8725) by lassebm
• Fixed Docker-based builder image. (#8718) by ilyam8
• Fixed status checks for UPS devices using the apcupsd collector. (#8688) by ilyam8
• Fixed the build matrix in the build and install GitHub Actions checks. (#8715) by Ferroin
• Fixed eBPF collector compatibility with the 7.x family of RedHat. (#8694) by thiagoftsm
• Fixed alarm notification script by adding a check to the Dynatrace notification method. (#8654) by ilyam8
• Fixed threads_creation_rate chart context in the python.d MySQL collector. (#8636) by ilyam8
• Fixed errors shown when running install-requred-packages.sh on certain Linux systems. (#8606) by ilyam8
• Fixed sudo check in charts.d libreswan collector to prevent daily security notices. (#8569) by ilyam8

Netdata v1.21.1

Release v1.21.1 is a hotfix release to improve the performance of the new React dashboard, which was merged and enabled by default in v1.21.0.

The React dashboard shipped in v1.21.0 did not properly freeze charts that were outside of the browser's viewport. If a user who loaded many charts by scrolling through the dashboard, charts outside of their browser's viewport continued updating. This excess of chart updates caused all charts to update more slowly than every second.

v.1.21.1 includes improvements to the way the Netdata dashboard freezes, maintains state, and restores charts as users scroll.

Netdata v1.21.0

Release v1.21.0 contains 2 new collectors, 3 new exporting connectors, 37 bug fixes, 46 improvements, and 25 documentation updates. We also made 26 bug fixes or improvements related to the upcoming release of Netdata Cloud.

At a glance

We added a new collector for Apache Pulsar, a popular open-source distributed pub-sub messaging system. We use Pulsar in our Netdata Cloud infrastructure (more on that later this month!), and are excited to start sharing metrics about our own Pulsar systems when the time comes. The Pulsar collector attempts to auto-detect any running Pulsar processes, but you can always configure the collector based on your setup.

Also new in v1.21 is a VerneMQ collector. We use the open-source MQ Telemetry Transport (MQTT) broker for Netdata Cloud as well. As with Pulsar, you can configure the VerneMQ collector to auto-detect your installation in just a few steps.

Our experimental exporting engine received significant updates with new connectors for Prometheus remote write, MongoDB, and AWS Kinesis Data Streams. You can now send Netdata metrics to more than 20 additional external storage providers for long-term archiving and deeper analysis. Learn more about the exporting engine in our documentation.

We upgraded our TLS compatibility to include 1.3, which applies to HTTPS for both Netdata's web server and streaming connections. TLS 1.3 is the most up-to-date version of the TLS protocol, and contains important fixes and improvements to ensure strong encryption. If you enabled TLS in the web server or streaming, Netdata attempts to use 1.3 by default, but you can also set the version and ciphers explicitly. Learn more in the documentation.

The Netdata dashboard has been completely re-written in React. While the look and behavior hasn't changed, these under-the-hood changes enable a suite of new features, UX improvements, and design overhauls. With React, we'll be able to work faster and better resource our talented engineers.

As part of the ongoing work to polish our eBPF collector tech preview, we've now proven the collector's performance is very good, and have vastly expanded the number of operating system versions the collector works on. Learn how to enable it in our documentation. We've also extensively stress-tested the eBPF collector and found that it's impressively fast given the depth of metrics it collects! Read up on our benchmarking analysis on GitHub.

Acknowledgments

• Jiab77 for helping remove extra printed \n in various installation methods.
• SamK for fixing missing folders in /var/ for .deb installations.
• kevenwyld for improving Netdata's support of RHEL distributions.
• WoozyMasta for adding in the ability to get Kubernetes pod names with kubectl in bare-metal deployments.
• paulmezz for adding the ability to to connect to non-admin user IDs when trying to collect metrics from a Ceph storage cluster.
• ManuelPombo for adding additional charts to our Postgres collector, and anayrat for helping review the changes.
• Default for adding lsyncd to the backup group in apps.plugin.
• bceylan, peroxy, toadjaune, grinapo, m-rey, and YorikSar for documentation fixes.

Breaking changes

None.

Improvements

• Extended TLS support for 1.3. (#8505) by thiagoftsm
• Switched to the React dashboard code as the default dashboard. (#8363) by Ferroin

Collectors

• Added a new Pulsar collector. (#8364) by ilyam8
• Added a new VerneMQ collector. (#8236) by ilyam8
• Added high precision timer support for plugins such as idlejitter. (#8441) by mfundul
• Added an alarm to the dns_query collector that detects DNS query failure. (#8434) by ilyam8
• Added the ability to get the pod name from cgroup with kubectl in bare-metal deployments. (#7416) by WoozyMasta
• Added the ability to connect to non-admin user IDs for a Ceph storage cluster. (#8276) by paulmezz
• Added connections (backend) usage to Postgres monitoring. (#8126) by ManuelPombo
• eBPF: Added support for additional Linux kernels found in Debian 10.2 and Ubuntu 18.04. (#8192) by thiagoftsm

Packaging/installation

• Added missing override for Ubuntu Eoan. (#8547) by prologic
• Added Docker build arguments to pass extra options to Netdata installer. (#8472) by Ferroin
• Added deferred error message handling to the installer. (#8381) by Ferroin
• Fixed cosmetic error checking for CentOS 8 version in install-required-packages.sh. (#8339) by prologic
• Added various fixes and improvements to the installers. (#8315) by Ferroin
• Migrated to installing only Python 3 packages during installation. (#8318) by Ferroin
• Improved support for RHEL by not installing the CUPS plugin when v1.7 of CUPS cannot be installed. (#7216) by kevenwyld
• Added support for Clear Linux in install-required-packages.sh. (#8154) by Ferroin
• Removed Fedora 29 from CI and packaging. (#8100) by Ferroin
• Removed Ubuntu 19.04 from CI and packaging. (#8040) by Ferroin
• Removed OpenSUSE Leap 15.0 from CI. (#7990) by Ferroin

Exporting

• Added a MongoDB connector to the exporting engine. (#8416) by vlvkobal
• Added a Prometheus Remote Write connector to the exporting engine. (#8292) by vlvkobal
• Added an AWS Kinesis connector to the exporting engine. (#8145) by vlvkobal

Documentation

• Fixed typo in main README.md. (#8547) by bceylan
• Updated the update instructions with per-method details. (#8394) by joelhans
• Updated paragraph on install-required-packages.sh. (#8347) by prologic
• Added Patti's dashboard video to the documentation. (#8385) by joelhans
• Fixed go.d modules in the COLLECTORS.md. (#8380) by ilyam8
• Added frontmatter to all documentation in bulk. (#8354) and (#8372) by joelhans
• Fixed MDX parsing in installation guide. (#8362) by joelhans
• Fixed typo in eBPF documentation. (#8360) by ilyam8
• Fixed links in packaging/installer to work on GitHub and docs. (#8319) by joelhans
• Fixed typo in main README.md. (#8335) by peroxy
• Removed mention saying that .deb packages are experimental. (#8250) by toadjaune
• Added standards for abbreviations/acronyms to docs style guide. (#8313) by joelhans
• Tweaked eBPF documentation, and added performance data. (#8261) by joelhans
• Added requirements for the exim collector. (#8096) by petarkozic
• Fixed misspelling of openSUSE and SUSE. (#8233) by m-rey
• Added OpenGraph tags to documentation pages. (#8224) by joelhans
• Fixed typo in custom dashboard documentation. (#8213) by shortpatti
• Removed extra asterisks in main README. (#8193) by grinapo
• Added eBPF README to documentation navigation and improved page title. (#8191) by joelhans
• Fixed figure+image without closing tag in new documentation. (#8177) by joelhans
• Corrected instructions for running Netdata behind Apache. (#8169) by cakrit
• Added PR title guidelines to the contribution guidelines to make CHANGELOG.md more meaningful. (#8150) by cakrit
• Fixed formatting in Custom dashboards documentation. (#8102) by YorikSar
• Updated the manual install documentation with better information about CentOS 6. (#8088) by Ferroin
• Added tutorials to support v1.20 release (#7943) by joelhans

CI/CD

• Added logic to bail early on LWS build if cmake is not present. (#8559) by Ferroin
• Added python.d configuration files to YAML linting CI process and increase line limit to 120 characters. (#8541) and (#8542) by ilyam8
• Cleaned up GitHub Actions workflows. (#8383) by Ferroin
• Migrated tests from Travis CI to Github Workflows. (#8331) by prologic
• Covered install-required-packages.sh with Coverity scan. (#8388) by prologic
• Added support for cross-host docker-compose builds. (#7754) by amoss
• Reconfigured Travis CI to retry transient failures on lifecycle tests. (#8203) by prologic
• Switched to checkout@v2 in GitHub Actions. (#8170) by ilyam8

Other

• Added lsyncd to the backup group in apps.plugin. (#8159) by Default

Netdata Cloud

• Fixed compiler warnings in the claiming code. (#8567) by vlvkobal
• Fixed regressions in cloud functionality (build, CI, claiming). (#8568) by underhood
• Switched over to soft feature flag. (#8545) by amoss
• Improved claiming behavior to run as netdata user by default, or override if necessary. (#8516) by amoss
• Updated the info endpoint for Cloud notifications. (#8519) by amoss
• Added correct error logging for ACLK challenge/response. (#8538) by stelfrag
• Cleaned up Cloud configuration files to move [agent_cloud_link] settings to [cloud]. (#8501) by underhood
• Enhanced ACLK header payload to include timestamp-offset-usec. (#8499) by stelfrag
• Added ACLK build failures to anonymous statistics. (#8429) by underhood
• Added ACLK connection failures to anonymous statistics. (#8456) by underhood
• Added HTTP proxy support to ACLK. (#8406)/(#8418) by underhood
• Improved ownership of the claim.d directory. (#8475) by amoss
• Fixed the ACLK response payload to match the new specification. (#8420) by stelfrag
• Added the new cloud info in the info endpoint. (#8430) by amoss
• Implemented ACLK Last Will and Testament. (#8410) by stelfrag
• Fixed JSON parsing in ACLK. (#8426) by stelfrag
• Fixed outstanding problems in claiming and add SOCKS5 support. (#8406)/(#8404) by amoss and underhood
• Fixed the type value for alarm updates in the ACLK. (#8403) by stelfrag
• Improved performance of ACLK. (#8399)/(#8401) by amoss
• Improved the ACLK's agent "pop-corning" phase. (#8398) by stelfrag
• Improved ACLK according to results of the smoke-test. (#8358) by amoss and underhood
• Added code to bundle LWS in binary packages. (#8255) by Ferroin
• Added libwebsockets files to make dist. (#8275) by Ferroin
• Adapted the claiming script to new API responses. (#8245) by hmoragrega
• Fixed claiming script to reflect Netdata Cloud API changes. (#8220) by cosmix
• Added libwebsockets bundling code to netdata-installer.sh. (#8144) by Ferroin

Bug fixes

• Removed notifications from the dashboard and fixed the /default.html route. (#8599 by jacekkolasa
• Fixed help-tooltips styling, private registry node deletion, and the right-hand sidebar "jumping" on document clicks. (#8553 by jacekkolasa
• Fixed errors reported by Coverity. (#8593) by thiagoftsm, (#8579) by amoss, and (#8586) by thiagoftsm
• Added netdata.service.* to .gitignore to hide system/netdata.service.v235 file. (#8556) by vlvkobal
• Fixed Debian 8 (Jessie) support. (#8590) and (#8593) by prologic
• Fixed broken Fedora 30/31 RPM builds. (#8572) by prologic
• Fixed broken pipe ignoring in apps.plugin. (#8554) by vlvkobal
• Fixed the bytespersec chart context in the Python Apache collector. (#8550) by ilyam8
• Fixed charts.d.plugin to exit properly during Netdata service restart. (#8529) by ilyam8
• Fixed minimist dependency vulnerability. (#8537) by jacekkolasa
• Fixed our Debian/Ubuntu packages to package the expected systemd unit files. (#8468) by prologic
• Fixed auto-updates for static (kickstart-static64.sh) installs. (#8507) by prologic
• Fixed openSUSE 15.1 RPM package builds. (#8494) by prologic
• Fixed how SimpleService truncates Python module names. (#8492) by ilyam8
• Removed erroneous \n in uninstaller output. (#8446) by prologic
• Fixed install-required-packages script to self-update apt. (#8491) by prologic
• Added proper prefix to Python module names during loading. (#8474) by ilyam8
• Fixed how the Netdata updater script cleans up after being run. (#8414) by prologic
• Fixed the flushing error threshold with the database engine. (#8425) by mfundul
• Fixed memory leak for host labels streaming from slaves to master. (#8460) by thiagoftsm
• Fixed support for uninstalling the eBPF collector in the uninstaller. (#8444) by prologic
• Fixed a bug involving stop_all_netdata uv_pipe_connect() in the installer. (#8444) by prologic
• Fixed installer output regarding newlines. (#8447) by prologic
• Fixed broken dependencies for Ubuntu 19.10. (#8397) by prologic
• Fixed streaming scaling. (#8375) by mfundul
• Fixed missing characters in kernel version field by encoding slave fields. (#8216) by thiagoftsm
• Fixed installation for Ubuntu 14.04 (#7690) by Ehekatl
• Fixed dependencies for Debian Jessie. (#8290) by Ferroin
• Fixed dependency names for Arch Linux. (#8334) by Ferroin
• Removed extra printed \n in various installers. (#8324)/(#8325)/(#8326) by Jiab77
• Fixed missing folders in /var/ for .deb packages. (#8314) by SamK
• Fixed Ceph collector to get osd_perf_infos in versions 14.2 and higher. (#8248) by ilyam8
• Fixed RHEL / CentOS 8.x dependencies for Judy-devel and others.(#8202) by prologic
• Removed extraneous commas from chart information in dashboard. (#8266) by FlyingSixtySix
• Removed tmem collection from xenstat_plugin to allow Netdata on Xen 4.13 to compile successfully. (#7951) by rushikeshjadhav
• Fixed get_latest_version for nightly channel update script. (#8172) by ilyam8
• Restricted messages to Google Analytics. (#8161) by thiagoftsm
• Fixed Python 3 dict access in OpenLDAP collector module. (#8162) by Mic92

Netdata v1.20.0

Release v1.20.0 contains 3 new collectors, 54 bug fixes, 89 improvements, and 38 documentation updates.

At a glance

Our first major release of 2020 comes with an alpha version of our new eBPF collector. eBPF (extended Berkeley Packet Filter) is a virtual bytecode machine, built directly into the Linux kernel, that you can use for advanced monitoring and tracing.

With this release, the eBPF collector monitors system calls inside your kernel to help you understand and visualize the behavior of your file descriptors, virtual file system (VFS) actions, and process/thread interactions. You can already use it for debugging applications and better understanding how the Linux kernel handles I/O and process management.

The eBPF collector is in a technical preview, and doesn't come enabled out of the box. If you'd like to learn more about_why_ eBPF metrics are such an important addition to Netdata, see our blog post: Linux eBPF monitoring with Netdata. When you're ready to get started, enable the
eBPF collector by following the steps in our documentation.

This release also introduces host labels, a powerful new way of organizing your Netdata-monitored systems. Netdata automatically creates a handful of labels for essential information, but you can supplement the defaults by segmenting your systems based on their location, purpose, operating system, or even when they went live.

You can use host labels to create alarms that apply only to systems with specific labels, or apply labels to metrics you archive to other databases with our exporting engine. Because labels are streamed from slave to master systems, you can now find critical information about your entire infrastructure directly from the master system.

Our host labels tutorial will walk you through creating your first host labels and putting them to use in Netdata's other features.

Finally, we introduced a new CockroachDB collector. Because we use CockroachDB internally, we wanted a better way of keeping tabs on the health and performance of our databases. Given how popular CockroachDB is right now, we know we're not alone, and are excited to share this collector with our community. See our tutorial on monitoring CockroachDB metrics for set-up details.

We also added a new squid access log collector that parses and visualizes requests, bandwidth, responses, and much more. Our apps.plugin collector has new and improved way of processing groups together, and our cgroups collector is better at LXC (Linux
container) monitoring.

Speaking of collectors, we revamped our collectors documentation to simplify how users learn about metrics collection. You can now view a collectors quickstart to learn the process of enabling collectors and monitoring more applications and services with Netdata, and see everything Netdata collects in our supported collectors list.

Acknowledgements

We're extremely grateful to the following contributors for their help since our last major release in November 2019. Whether it's their first or fiftieth contribution, insights from our users not only help make Netdata better, but also remind us why we're so lucky to be part of a vibrant open-source community.

• k0ste and DefauIt for improving the application groups of the apps plugin.
• gmeszaros for a fix to the broken updater.
• blaines for an elastisearch collector fix.
• stevenh for adding freeipmi support to our Docker image and lassebm for related fixes and documentation.
• yasharne for helping us improve the httpcheck collector.
• candrews for the introduction of -fno-common in CFLAGS.
• Jiab77 for fixing a typo in the installer options.
• amishmm for improvements to the systemd service files.
• tnyeanderson for continuing to improve his multi-host sample dashboard.
• yasharne and especially schneiderl for corrections to the docs.
• lucasRolff for improvements to the litespeed collector.
• Ehekatl for the improvements to the Prometheus remote write API and the fix to thesoftnet alarm.
• wonsangki for translating several docs into Korean.
• candrews for fixing the option to disable the Prometheus remote API from configure.
• kkoomen for improvements to the Apache proxy guide.
• vzDevelopment for assistance with the unicode support in the python.d plugin.
• hexchain for the addition of pressure stall information to the proc plugin.
• nabijaczleweli and rex4539 for documentation fixes.

Breaking Changes

• Removed deprecated bash collectors apache, cpu_apps, cpufreq, exim, hddtemp, load_average, mem_apps, mysql, nginx, phpfpm, postfix, squid, tomcat #7962 (ilyam8). If you were still using one of these collectors with custom configurations, you can find the new collector that replaces it in the supported collectors list.
• Modified the Netdata updater to prevent unecessary updates right after installation and to avoid updates via local tarballs #7939 (prologic). These changes introduced a critical bug to the updater, which was fixed via #8057 #8076 (prologic) and #8028 (gmeszaros). See issue 8056 if your Netdata is stuck on v1.19.0-432.

Improvements

Host Labels

• Added support for host labels #7515 #7449 (amoss)
• Improved the monitored system information detection. Added CPU freq & cores, RAM and disk space. #7815 #7866 (Ferroin), #7862 (thiagoftsm)
• Started distinguishing the monitored system's (host) OS/Kernel etc. from those of the docker container's #7770 (amoss)
• Started creating host labels from collected system info #7485 (vlvkobal)
• Started passing labels and container environment variables via the streaming protocol#7549 #8011 (thiagoftsm)
• Started sending host labels via exporting connectors #7554 #7702(vlvkobal)
• Added label support to alarm definitions and started recording them in alarm logs #7548 #7594#7462 #7600 (thiagoftsm)
• Added support for host labels to the API responses #7493 #7616 (vlvkobal)
• Added configurable host labels to netdata.conf #7451 #7458 (thiagoftsm)
• Added kubernetes labels #7510 #7453 (cakrit)

New Collectors

• eBPF kernel collector #7979 (thiagoftsm) #8075 (prologic)
• CockroachDB (go.d.plugin #322)
• squidlog: squid access log parser (go.d.plugin #304)

Collector improvements

• apps.plugin
  • Created dns group. #8058 (k0ste)
  • Improved database group. #8004 (DefauIt)
  • Improved ceph & samba groups. #7982 (k0ste)
• varnish: Added SMF metrics (cache on disk) #7926 (ilyam8)
• phpfpm: Fixed per process chart titles and readme #7876 (ilyam8)
• python.d: Formatted the code in all modules #7832 (ilyam8)
• node.d/snmp: - Added snmpv3 support #7802 (ilyam8) - Formatted the code in snmp.node.js #7816 (ilyam8)
• cgroups: Improved LXC monitoring by filtering out irrelevant LXC cgroups #7760 (vlvkobal)
• litespeed: Added support for different .rtreport format #7705 (lucasRolff)
• freeipmi: Added support to the docker image #7081 (stevenh)
• proc.plugin: Added pressure stall information #7209 #7547 (hexchain)
• sensors: Improved collection logic #7447 (ilyam8)
• proc: Started monitoring network interface speed, duplex, operstate #7395 (stelfrag)
• smartd_log: Fixed the setting in the reallocated sectors count, by setting ATTR5 chart algorithm to absolute #7384 (ilyam8)
• nvidia-smi: Allow executing nvidia-smi in normal instead of loop mode #7372 (ilyam8)
• wmi: collect logon metrics, collect logical_disk disk latency metrics
• weblog: handle MKCOL, PROPFIND, MOVE, SEARCH http request methods
• scaleio: storage pools and sdcs metrics. (#294)

Exporting Engine

• Implemented the main flow for the Exporting Engine #7149 (vlvkobal)

Streaming

• Add versioning to the streaming protocol #7851 (thiagoftsm)

Installation/Packages

• Fixed missing directory when creating the symbolic link during eBPF installation and remove future options. #8133 (prologic)
• Fixed NetData installer on *BSD systems after libmosquitto and eBPF functionality was enabled. #8121 (prologic)
• Fixed issues with the RPM nightly builds resulting from the bundled libmosquitto functionality that was recently merged. #8109 (Ferroin)
• Corrected the invocations of mktemp so that they produce temporary directories in $TEMPDIR instead of the current directory, in a way that is compatible with busybox. #8066 (Ferroin)
• Improved CI/CD workflow to install required packages and build the agent across all the OS/Distro(s) we support #7969 #7949 (prologic)
• Updated the installer to download go.d.plugin, only if we have a new version #7946 (ilyam8)
• Assorted cleanup items in the RPM spec file. #7927 (Ferroin)
• Added a new, simpler, Alpine based Dockerfile for quick dev and testing #7914 (prologic)
• Added minor fixes and improvements to the installer/updater shell scripts. #7847 (prologic)
• Added ReviewDog CI checks
  • JavaScript #7828 (prologic)
  • Golang #7827 (prologic)
  • Shell scripts in PRs #7795 (prologic)
• Stopped removing netdata groups/users during uninstall (Debian postrm) #7817 (prologic)
• Started using the system service manager to shut down Netdata. #7814 (Ferroin)
• Improved the systemd service files, by removing unecessary ExecStartPre lines and moving global options to netdata.conf #7790 (amishmm)
• Removed unnessecary echo calls from the updater. #7783 (Ferroin)
• Fixed warnings in the Debian package build process and enabled the builds to work with older versions of dpkg-buildpackage by modifying the formatting of the trailer line in the Debian changelog template. #7763 (Ferroin)
• Cleaned up static build process, by using /bin/sh and removing use of sudo #7725 (prologic)
• Added auto-updates to kickstart-static64 installations. #7704 (Ferroin)
• Added static build support for Prometheus remote write #7691 (Ehekatl)
• Moved the script for installing required packages into the main repo. #7563 (Ferroin)
• Updated the distribution support matrix. #7636 (Ferroin)
• Added Ubuntu 19.10 to packaging and lifecycle checks. #7629 (Ferroin
• Removed EOL distros from CI jobs. #7628 (Ferroin)
• Made the netdata installer more flexible, to accommodate install with ssl on MacOS #6922 (paulkatsoulakis)
• Improved shutdown of the Netdata agent on update and uninstall. #7595 (Ferroin)
• Added Fedora 31 CI integrations. #7524 (Ferroin)
• Removed CentOS 6 package building and lifecycle tests #7425 (knatsakis), #7430 (ncmans)
• Removed -f option from groupdel in uninstaller. #7507 (Ferroin)
• Injected archived backports repository on Debian Jessie for CI package builds. #7495 (Ferroin)
• Set the default release channel to stable #7399 (ncmans)
• Removed EOL'd Ubuntu Trusty (14.04) from build #7481 (ncmans)
• Corrected installer instructions during a non-privileged install #7393 (julidegulen)

Documentation

• Added the step-by-step Netdata tutorial #7489 (joelhans)
• Overhauled the installation documentation #7841 (joelhans)
• Refactored the collectors documentation #8074 (shortpatti), #8086 #8052 #7996 (joelhans), #8009 #8005 #7997 (ilyam8)
• Restructured the health documentation #7329 (joelhans)
• Promoted DB engine/long-term metrics storage more heavily and fix misleading information #8031 (joelhans), #8017 (underhood)
• Updated eBPF docs with better install/enable instructions #8125 (joelhans)
• Allowed parentheses in heading links #7995 (joelhans)
• Fixed typos in the tutorial #7978 (joelhans)
• Indicated FreeIPMI supported in Docker image #7964 (lassebm)
• Fixed wrong code fragments in signing in to the cloud instructions #7950 (cakrit)
• Fixed variety of linter errors across docs #7944 #7526 #7407 (joelhans)
• Cleanup of macOS installation docs #7925 (joelhans)
• Fixed typo in PULL_REQUEST_TEMPLATE #7924 (joelhans)
• Added doc with post-install instructions for Google Cloud Platform #7912 (joelhans)
• Clarify the rules to create an alarm name #7911 (thiagoftsm)
• Added docs about using caching proxies with our package repos. #7909 (Ferroin)
• Added docs for how to build/install NetData on CentOS 8.x #7890 (prologic)
• Clarified editing health config files in health quickstart #7883 (joelhans)
• Added retroshare collector readme #7849 (ilyam8)
• Fixed typo in the SSV formatter documentation #7782 (cosmix)
• Added a missing parameter to the allmetrics endpoint documentation #7776 (vlvkobal)
• Documented how to fix the width of badges #7764 (underhood)
• Improved styling of documentation site and stared using Algolia search #7753 (joelhans)
• Fixed typos in docs #7752 (schneiderl), #7737 (yasharne)
• Added Korean translation of some files to docs netdata/localization issue 25 (wonsangki), #7723 (cakrit)
• Added better control for the introduction of new languages in docs translations #7722 (cakrit)
• Added a Dockerfile.docs to easily and build/rebuild docs #7688 (prologic)
• Corrected pfSense installation instructions #7665 (prologic)
• Fixed buildyaml.sh script so that docs generation works correctly. #7662 (Ferroin
• Fixed to new health documentation structure #7419 (joelhans)
• Changed build process to allow apostrophes in headers #7431 (joelhans)
• Added configuration details for vhost about DOSPageCount to Apache proxy guide #7582 (kkoomen)
• Added notice about mod_evasive to Apache proxy guide #7578 (joelhans)
• Fixed broken docs builds #7409 (joelhans)
• Fixed linter errors in packaging/docker/README #7199 (joelhans)
• Updated the python.d README #7357 (OdysLam)
• Documented per-chart configuration options #7345 (joelhans)
• Fixed typos and markup #7368 (nabijaczleweli), #7375 (rex4539)
• Fixed errors in plugins.d/README.md #7340 (joelhans)

Privacy

• Added support for opting out of telemetry via the DO_NOT_TRACK environment variable #7846 #7929 (prologic)
• Fixed typo in the installer options to disable telemetry #7843 (Jiab77)
• Improved documentation of opting out of anonymous statistics #7597 (joelhans)
• Added anon tracking notice for installers #7437 (ncmans)

Other

• Preparations for the next netdata cloud release. Added custom libmosquitto, netdata-cli and other prerequisites:
  • #8085 #8067 #8025 #8047 #7592 #7513 (Ferroin)
  • #7894 #7682 (stelfrag)
  • #7836 (thiagoftsm)
  • #8030 #7988 #7713 (underhood)
  • #7750 (jacekkolasa)
  • #7525 (mfundul)
  • #7444 (amoss)
• Improved the GitHub labeler. #8071 #8032 (ilyam8), #7543 #7768 #7699 #7697 #7630 #7699 #7697 (Ferroin)
• Added testing section to the PR template. #8068 (amoss)
• Applied linter fixes in shell scripts #7937 #7932 #7915 (prologic)
• Started supporting -fno-common in CFLAGS #7870 (#7877 (thiagoftsm)
• Completely removed the unbound python collector (dead code) #7853 (ilyam8)
• Added possibility to change badges' text font color #7809 (underhood)
• Small updates to sample multi-host dashboard, dash.html #7757 (tnyeanderson)
• Added missing quoting in shell scripts. #7685 (Ferroin)
• Bump handlebars from 4.2.0 to 4.5.3 #7654 (dependabot[bot]
• Reduce log level for uv_thread_set_name_np from error to info. #7653 (Saruspete
• Added sample cmds to get OS env in GitHub issue templates #7550 (Saruspete)
• Set standard name to non-libnetdata threads (libuv, pthread) (#7584 (Saruspete)

Bug fixes

• Fixed problems reported by Coverity for eBPF collector plugin. #8135 (thiagoftsm)
• Fixed invalid literal for float\(\): NN.NNt error in the elasticsearch python plugin, by adding terabyte unit parsing. #8013 (blaines)
• Fixed timeout failing in docker containers which broke some python.d collectors #8002 (ilyam8)
• Fixed python collectors to work on synology6 #7980 (ilyam8)
• Fixed problem with the httpcheck python collector not being able to check URLs with the POST method, by adding body to the URLService #7956 (ilyam8). Also record the new options in httpcheck.conf #7952 (yasharne)
• Fixed netdata-updater.sh appearing to fail #7955 (ilyam8)
• Fixed error/warnings found by shellcheck for the netdata-updater.sh #7938 (prologic)
• Fixed editing configuration via edit-config, when NetData is installed to a symlinked /opt #7933 (prologic)
• Fixed installation failures due to .keep files #7829 (prologic)
• Fixed installation on FreeBSD systems with non GNU sed #7796 (prologic)
• Fixed Source0 URL in RPM spec #7794 (prologic)
• Fixed text if current version is >= latest version and already installed #8078 (prologic)
• Fixed CentOS 7 RPM build failures. #7993 (Ferroin)
• Fixed wrong messages during the build process #7989 (Ferroin)
• Fixed the unit tests for the exporting engine #7784 (vlvkobal)
• Fixed a Coverity issue with an unchecked return value #7780 (vlvkobal)
• Fixed port in use after uninstall issue, by resolving a libuv IPC pipe cleanup problem #7778 (mfundul)
• Fixed dbengine repeated global flushing errors and collectors being blocked, by dropping dirty dbengine pages if the disk cannot keep up #7777 (mfundul)
• Fixed issue with alarm notifications occasionally ignoring the configured severity filter when the ROLE was set to root. #7769 (thiagoftsm)
• Fixed Netlink Connection Tracker charts in the nfacct plugin #7727 (vlvkobal)
• Fixed support for read-only /lib on SystemD systems like CoreOS in static build installation #7726 (prologic)
• Fixed invalid shell installer error and netdata not starting from its installed location. #7698 (Ferroin)
• Fixed metric values sent via remote write to Prometheus backends, when using average/sum #7694 (Ehekatl)
• Fixed unclosed brackets in softnet alarm #7693 (Ehekatl)
• Fixed SEGFAULT when localhost initialization failed #7663 (underhood)
• Fixed the handling of permissions in the installer script and the RPM spec file so that theya re consistent with each other and with a clean install done with make install. #7632 (Ferroin)
• Reduced the number of broken pipe error log entries, after a SIGKILL #7588 (thiagoftsm)
• Fixed a syntax error in the packaging functions. #7686 (Ferroin)
• Fixed Coverity errors by restoring support for protobuf 3.0 #7683 (vlvkobal)
• Fixed inability to disable Prometheus remote API #7674 (candrews)
• Fixed SEGFAULT from the cpuidle plugin #7664 (Saruspete)
• Fixed samba collector not working, due to inability to run sudo #7655 (ilyam8)
• Fixed invalid css/js resource errors when URL for slave node has no final / on streaming master #7643 (underhood)
• Fixed keys_redis chart in the redis collector, by populating keys at runtime #7639 (ilyam8)
• Fixed UrlService bytes decoding and logger unicode encoding in the python.d plugin #7601 #7614 (ilyam8), #7376 (vzDevelopment)
• Fixed a warning in the prometheus remote write backend #7609 (vlvkobal)
• Fixed not detecting more than one adapter in the hpssa collector #7580 (gnoddep)- Fixed race condition in dbengine #7565 (thiagoftsm)
• Fixed race condition with the dbenging page cache descriptors #7478 (mfundul)
• Fixed dbengine dirty page flushing warning #7469 (mfundul)
• Fixed missing parenthesis on alarm softnet.conf #7476 (Steve8291)
• Fixed race condition in the dbengine #7533 (mfundul)
• Fixed "Master thread EXPORTING takes too long to exit. Giving up" error, by cleaning up the main exporting engine thread on exit #7558 (vlvkobal)
• Fixed rabbitmq error "update() unhandled exception: invalid literal for int() with base 10" #7464 (ilyam8)
• Fixed some LGTM alerts #7441 (jacekkolasa)
• Fixed valgrind errors #7532 (mfundul)
• Fixed monit collector LGTM warnings (#7387 (ilyam8)
• Fixed the following go.d.plugin collector issues: - mysql: panic in Cleanup (#326) - unbound: gather metrics via unix socket (#319) - logstash: pipelines chart (#317) - unbound: configuration file parsing. Support include mechanism. (#298) - logstash: pipelines metrics parsing (#293) - phpfpm: processes metrics parsing (#297)

Netdata v1.19.0

Release v1.19.0 contains 2 new collectors, 19 bug fixes, 17 improvements, and 19 documentation updates.

At a glance

We completed a major rewrite of our web log collector to dramatically improve its flexibility and performance. The new collector, written entirely in Go, can parse and chart logs from Nginx and Apache servers, and combines numerous improvements. Netdata now supports the LTSV log format, creates charts for TLS and cipher usage, and is amazingly fast. In a test using SSD storage, the collector parsed the logs for 200,000 requests in about 200ms, using 30% of a single core.

This Go-based collector also has powerful custom log parsing capabilities, which means we're one step closer to a generic application log parser for Netdata. We're continuing to work on this parser to support more application log formatting in the future.

We have a new tutorial on enabling the Go web log collector and using it with Nginx and/or Apache access logs with minimal configuration. Thanks to Wing924 for starting the Go rewrite!

We introduced more cmocka unit testing to Netdata. In this release, we're testing how Netdata's internal web server processes HTTP requests—the first step to improve the quality of code throughout, reduce bugs, and make refactoring easier. We wanted to validate the web server's behavior but needed to build a layer of parametric testing on top of the CMocka test runner. Read all about our process of testing and selecting cmocka on our blog post: Building an agile team's 'safety harness' with cmocka and FOSS.

Netdata's Unbound collector was also completely rewritten in Go to improve how it collects and displays metrics. This new version can get dozens of metrics, including details on queries, cache, uptime, and even show per-thread metrics. See our tutorial on enabling the new collector via Netdata's amazing auto-detection feature.

We fixed an error where invalid spikes appeared on certain charts by improving the incremental counter reset/wraparound detection algorithm.

Netdata can now send health alarm notifications to IRC channels thanks to Strykar!

And, Netdata can now monitor AM2320 sensors, thanks to hard work from Tom Buck.

Acknowledgements

Our thanks go to:

• andyundso for fixing the packagecloud binary installation in Debian 8.
• Strykar for adding support IRC health notifications.
• tommybuck for the new AM2320 sensors collector.
• Saruspete for the new ability to provide metrics on fragmentation of free memory pages.
• OdysLam for improving the documentation for new collector plugins.
• k0ste, xginn8 and nodiscc for improving the configuration of the apps plugin.
• amichelic for improving the web_log collector.
• cherouvim, arkamar, half-duplex and CtrlAltDel64 for improving the documentation.
• mniestroj for the fix to the dbengine compilation with musl standard C.
• arkamar for an improvement to the xenstat collector.
• vakartel for improving the cgroup network interfaces detection in Proxmox 6.

Improvements

New Collectors

• AM2320 sensor collector plugin #7024 (tommybuck)
• Added parsing of /proc/pagetypeinfo to provide metrics on fragmentation of free memory pages. #6843 (Saruspete)
• The unbound collector module was completely rewritten, in Go go.d.plugin/#287 (ilyam8)

Collector improvements

• We rewrote our web log parser in Go, drastically improving its flexibility and performance. go.d.plugin/#141 (ilyam8)
• The Kubernetes kubelet collector now reads the service account token and uses it for authorization. We also added a new default job to collect metrics from https://localhost:10250/metrics. go.d.plugin/#285
• Added a new default job to the Kubernetes coredns collector to collect metrics from http://kube-dns.kube-system.svc.cluster.local:9153/metrics. go.d.plugin/#285
• apps.plugin: Synced FRRouting daemons configuration with the frr 7.2 release. #7333 (k0ste)
• apps.plugin: Added process group for git-related processes. #7289 (nodiscc)
  -apps.plugin: Added balena to the container-engines application group. #7287 (xginn8)
• web_log: Treat 401 Unauthorized requests as successful. #7256 (amichelic)
• xenstat.plugin: Prepare for xen 4.13 by checking for check xenstat_vbd_error presence. #7103 (arkamar)
• mysql: Added galera cluster_status alarm. #6989 (ilyam8)

Metrics Database

• Netdata generates alarms if the disk cannot keep up with data collection. #7139 (mfundul)

Health

• Fine tune various default alarm configurations. #7322 (Ferroin)
• Update SYN cookie alarm to be less aggressive. #7250 (Ferroin)
• Added support for IRC alarm notifications #7148 (Strykar)

Installation/Packages

• Corrected the Makefile.am files indentation, to prevent unexpected errors. #7252 (knatsakis)
• Rationalized ownership and permissions of /etc/netdata. #7244 (knatsakis)
• Made various improvements to the installer script netdata-installer.sh. #7200 (knatsakis)
• Include go.d.plugin version v0.11.0 #7365 (ilyam8)

Documentation

• Correct versions of FreeNAS that Netdata is available on. #7355 (knatsakis)
• Update plugins.d/README.md. #7335 (OdysLam)
• Note regarding stable vs nightly was accidentally being shown as a code fragment in the installation documentation. #7330 (cakrit)
• Properly link to translated documents from netdata-security.md. #7343 (cakrit)
• Update documentation of the netdata-updater, to properly cover kickstart-static64.sh and kickstart.sh installations. #7262 (knatsakis)
• Converted the swagger documentation to OpenAPI3.0. #7257 (amoss)
• Minor corrections to the netdata installer documentation. #7246 (paulkatsoulakis)
• Fix typo in collectors README. #7242 (cherouvim)
• Clarified database engine/RAM in getting started guide. #7225 (joelhans)
• Suggest using /var/run/netdata for the unix socket, in running behind nginx documentation. #7206 (CtrlAltDel64)
• Added GA links to new documents. #7194 (joelhans)
• Added a page for metrics archiving to TimescaleDB. #7180 (joelhans)
• Fixed typo in the contrib/debian descriptions for cupsd. #7154 (arkamar)
• Added user information to MySQL Python module documentation. #7128 (prhomhyse)
• Document the results of the spike investigation into CMake. #7114 (amoss)
• Fix to docker-compose+Caddy installation. #7088 (joelhans)
• Fixed broken links and added setup instructions for Telegram health notifications. #7033 (half-duplex)
• Minor grammar change in /web/gui documentation #7363 (eviemsrs)

Other

• Improve Travis build warnings (issue #7189). #7312 (amoss)
• cmocka testing for http requests #7308, #7308, #7264 #7210 (amoss and vlvkobal)
• CI/CD: Prevented nightly jobs from timing out #7238, #7214 (knatsakis)

Bug fixes

• Fixed packagecloud binary installation in Debian 8. #7342 (andyundso)
• Fixed missing libraries in certain compilations, by adding missing trailing backslash to Makefile.am. #7326 (oxplot)
• Prevented freezes due to isolated CPUs. #7318 (stelfrag)
• Fixed missing streaming when slave has SSL activated. #7306 (thiagoftsm)
• Fixed error 421 in IRC notifications, by removing a line break from the message. #7243 (thiagoftsm)
• proc/pagetypeinfo collection could under particular circumstances cause high CPU load. As a workaround, we disabled pagetypeinfo by default. #7230 (vlvkobal)
• Fixed incorrect memory allocation in proc plugin’s pagetypeinfo collector. #7187 (thiagoftsm)
• Eliminated cached responses from the postgres collector. #7228 (ilyam8)
• rabbitmq: Fixed "disk_free": "disk_free_monitoring_disabled" error. #7226 (ilyam8)
• Fixed build with musl standard C library by including limits.h before using LONG_MAX. #7224 (mniestroj)
• Fixed Apache module not working with letsencrypt certificate by allowing the python UrlService to skip tls_verify for http scheme. #7223 (ilyam8)
• Fixed invalid spikes appearing in certain charts, by improving the incremental counter reset/wraparound detection algorithm. #7220 (mfundul)
• Fixed DNS-lookup performance issue on FreeBSD. #7132 (amoss)
• Fixed handling of the stable option, so that the installers and automatic updater respect it. #7083 (knatsakis), #7051 (oxplot)
• Fixed handling of the static binary installer’s handling of the --auto-update option. #7076 (knatsakis)
• Fixed cgroup network interfaces classification on Proxmox 6. #7037 (vakartel)
• Added missing dbengine flags to the installer. #7027 (paulkatsoulakis)
• Fixed issue with unknown variables in alarm configuration expressions always being evaluated to zero. #6984 (thiagoftsm)
• Fixed issue of automatically picking up Pi-hole stats from a Pi-hole instance installed on another device by disabling the default job that collects metrics from http://pi.hole. go.d.plugin 289 (ilyam8)

Netdata v1.18.1

Release v1.18.1 contains 17 bug fixes, 5 improvements, and 5 documentation updates.

At a glance

Patch release 1.18.1 contains several bug fixes, mainly related to FreeBSD and the binary package generation process.

Netdata can now send notifications to Google Hangouts Chat!

On certain systems, the slabinfo plugin introduced in v1.18.0 added thousands of new metrics. We decided the collector's usefulness to most users didn't justify the increase in resource requirements. This release disables the collector by default.

Finally, we added a chart under Netdata Monitoring to present a better view of the RAM used by the database engine (dbengine). The chart doesn't currently take into consideration the RAM used for slave nodes, so we intend to add more related charts in the future.

Acknowledgements

We'd like to thank:

• hendrikhofstadt for the Google Hangouts notifications
• stevenh for the awesome zombie process reaper and the fix for the freeipmi collector
• samm-git for the addition of the VMware VMXNET3 driver to the default interfaces list for FreeBSD
• sz4bi for a documentation fix

Improvements

• Disable slabinfo plugin by default to reduce the total number of metrics collected #7056 (vlvkobal)
• Add dbengine RAM usage statistics #7038 (mfundul)
• Support Google Hangouts chat notifications #7013 (hendrikhofstadt)
• Add CMocka unit tests #6985 (vlvkobal)
• Add prerequisites to enable automatic updates for installations via the static binary (kickstart-static64.sh) #7060 (knatsakis)

Documentation

• Fix typo in health_alarm_notify.conf #7062 (sz4bi)
• Fix BSD/pfSense documentation #7041 (thiagoftsm)
• Document the structure of the api/v1/data API responses. #7012 (amoss)
• Tutorials to support v1.18 features #6993 (joelhans)
• Fix broken links in docs #7123 (joelhans)

Bug fixes

• Fix unbound collector timings: Convert recursion timings to milliseconds. #7121 (Ferroin)
• Fix unbound collector unhandled exceptions #7112 (ilyam8)
• Fix upgrade path from v1.17.1 to v1.18.x for deb packages #7118 (knatsakis)
• Fix CPU charts in apps plugin on FreeBSD #7115 (vlvkobal)
• Fix megacli collector binary search and sudo check #7108 (ilyam8)
• Fix missing packages, by running the triggers for DEB and RPM package build in separate stages #7105 (knatsakis)
• Fix segmentation fault in FreeBSD when statsd is disabled #7102 (vlvkobal)
• Fix Clang warnings #7090 (thiagoftsm)
• Fix python.d error logging: change chart suppress msg level from ERROR to INFO #7085 (ilyam8)
• Fix freeipmi update frequency check: was warning that 5 was too frequent and it was setting it to 5. #7078 (stevenh)
• Fix alarm configurations not getting loaded, via better handling of chart names with special characters #7069 (thiagoftsm)
• Fix dbengine not working when mmap fails - mostly with BSD kernels #7065 (mfundul)
• Fix FreeBSD issue due to incorrect size of a zeroed block #7061 (vlvkobal)
• Don't write HTTP response 204 messages to the logs #7035 (vlvkobal)
• Fix build when CMocka isn't installed #7129 (vlvkobal)
• FreeBSD plugin: Add VMware VMXNET3 driver to the default interfaces list #7109 (samm-git)
• Prevent zombie processes when a child is re-parented to netdata when its running in a container , by adding child process reaper #7059 (stevenh)

Netdata v1.18.0

Release v1.18.0 contains 5 new collectors, 19 bug fixes, 28 improvements, and 20 documentation updates.

At a glance

The database engine is now the default method of storing metrics in Netdata. You immediately get more efficient and configurable long-term metrics storage without any work on your part. By saving recent metrics in RAM and "spilling" historical metrics to disk for long-term storage, the database engine is laying the foundation for many more improvements to distributed metrics.

We even have a tutorial on switching to the database engine and getting the most from it. Or, just read up on how performant the database engine really is.

Both our python.d and go.d plugins now have more intelligent auto-detection by periodically dump a list of active modules to disk. When Netdata starts, such as after a reboot, the plugins use this list of known services to re-establish metrics collection much more reliably. No more worrying if the service or application you need to monitor starts up minutes after Netdata.

Two of our new collectors will help those with Hadoop big data infrastructures. The HDFS and Zookeeper collection modules come with essential alarms requested by our community and Netdata's auto-detection capabilities to keep the required configuration to an absolute minimum. Read up on the process via our HDFS and Zookeeper tutorial.

Speaking of new collectors—we also added the ability to collect metrics from SLAB cache, Gearman, and vCenter Server Appliances.

Before v1.18, if you wanted to create alarms for each dimension in a single chart, you need to write separate entities for each dimension—not very efficient or user-friendly. New dimension templates fix that hassle. Now, a single entity can automatically generate alarms for any number of dimensions in a chart, even those you weren't aware of! Our tutorial on dimension templates has all the details.

v1.18 brings support for installing Netdata on offline or air-gapped systems. To help users comply with strict security policies, our installation scripts can now install Netdata using previously-downloaded tarball and checksums instead of downloading them at runtime. We have guides for installing offline via kickstart.sh or kickstart-static64.sh in our installation documentation. We're excited to bring real-time monitoring to once-inaccessible systems!

Acknowledgements

Our thanks go to:

• Saruspete for several contributions, including the new slabinfo collector, that monitors SLAB cache mechanism metrics.
• agronick for the new Gearman worker statistics
  collector
• OneCodeMonkey for a bug fix in the alarm notification script.
• lets00 for providing a Portuguese (Brazil) translation of the installation instructions
• mbarper and davent for improvements to the uninstaller.
• n0coast for a documentation fix.

Improvements

Database engine

• Make dbengine the default memory mode #6977 (mfundul)
• Increase dbengine default cache size #6997 (mfundul)
• Reduce overhead during write IO #6964 (mfundul)
• Detect deadlock in dbengine page cache #6911 (mfundul)
• Remove hard cap from page cache size to eliminate deadlocks. #7006 (mfundul)

New Collectors

• SLAB cache mechanism (Saruspete)
• Gearman worker statistics
• vCenter Server Appliance
• Zookeeper servers
• [Hadoop Distributed File System (HDFS) nodes] (https://docs.netdata.cloud/collectors/go.d.plugin/modules/hdfs/)

Collector improvements

• rabbitmq: Add vhosts message metrics from /api/vhosts #6976 (ilyam8)
• elasticsearch: collect metrics from _cat/indices #6965 (ilyam8)
• mysql: collect galera cluster metrics #6962 (ilyam8)
• Allow configuration of the python.d launch command from netdata.conf #6781 (amoss)
• x509check: smtp cert check support (https://github.com/netdata/go.d.plugin/pull/261)
• dnsmasq_dhcp: respect conf-dir,conf-file,dhcp-host options (https://github.com/netdata/go.d.plugin/pull/268)
• plugin: respect previously running jobs after plugin restart (https://github.com/netdata/netdata/issues/6499)
• httpcheck: add current state duration chart (https://github.com/netdata/go.d.plugin/pull/270 )
• springboot2: fix context (https://github.com/netdata/go.d.plugin/pull/263)

Health

• Enable alarm templates for chart dimensions #6560 (thiagoftsm)
• Center the chart on the proper chart and time whenever an alarm link is clicked #6391 (thiagoftsm)

Installation/Packages

• netdata/installer: Add support for offline installations using kickstart.sh or kickstart-static64.sh #6693 (paulkatsoulakis)
• Allow netdata service installation, when docker runs systemd #6987 (paulkatsoulakis)
• Make spec file more consistent with version dependencies #6948 (paulkatsoulakis)
• Fix broken links on web files, for DEB #6930 (paulkatsoulakis)
• Introduce separate CUPS package for DEB #6724 and RPM #6700 distributions. (paulkatsoulakis). Do not build CUPS plugin subpackage on CentOS 6 and CentOS 7 #6926 (knatsakis)
• Various Improvements in the package release CI/CD flow #6914 #6905 #6842 #6837 #6838 #6834 (paulkatsoulakis), #6900 (cakrit)
• Remove RHEL7 - i386 binary distribution, until bug #6849 is resolved #6902 (paulkatsoulakis)
• Bring on board two scripts that build libuv and judy from source #6850 (paulkatsoulakis)

Documentation

• Add Portuguese (Brazil) translation of the installation instructions #16(lets00), #7004 (cakrit)
• Fix broken links found via linkchecker #6983 (joelhans)
• Clarification on configuring notification recipients #6961 (cakrit)
• Fix Remark Lint for READMEs in database #6942, contrib #6921, daemon README #6920 and backends #6917 (prhomhyse)
• Suggest using /run or /var/run for the unix socket #6916 (cakrit)
• Improve documentation for the SNMP collector #6915 (cakrit)
• Update docs for offline install #6884 (paulkatsoulakis)
• Remove Dollar sign from Bash code in documentation and fix remark-lint warnings #6880 (prhomhyse)
• Markdown syntax fixes for MDX parser #6877 (joelhans)
• Update python.d module checklist to match the current paths and build system. #6874 (Ferroin)
• Add instructions for simple SMTP transport #6870 (cakrit)
• Add example for prometheus archiving source parameter #6869 (cakrit)
• Fix broken links in the standard web dashboard doc #6854 (prhomhyse)
• Overhaul of Getting started guide #6811 (joelhans)
• NPM Packages version update #6801 (prhomhyse)
• Update suggested grep command in “high performance netdata” to be more specific #6794 (n0coast)

Other

• API: Include family into the allmetrics JSON response #6966 (ilyam8)
• API: Add fixed width option to badges #6903 (underhood)
• Allow hostnames in Access Control Lists #6796 (amoss)
• Functional test improvements for web and alarms tests #6783 (thiagoftsm)

Bug fixes

• Fix issue error in alarm notification script, when executed without any arguments #7003 (OneCodeMonkey)
• Fix Coverity warnings #6992 #6970 #6941 #6797 (thiagoftsm), #6909 (cakrit)
• Fix dbengine consistency when a writer modifies a page concurrently with a reader querying its metrics #6979 (mfundul)
• Fix memory leak on netdata exit #6945 (vlvkobal)
• Fix for missing boundary data points in certain cases #6938 (mfundul)
• Fix unhandled exception log warnings in the python.d collector orchestrator start\_job #6928 (ilyam8)
• Fix CORS errors when accessing the health management API, by permitingt x-auth-token in Access-Control-Allow-Headers #6894 (cakrit)
• Fix misleading error log entries RRDSET: chart name 'XXX' on host 'YYY' already exists, by changing the log level for chart updates #6887 (vlvkobal)
• Properly resolve all Kubernetes container names #6885 (cakrit)
• Fix LGTM warnings #6875 (jacekkolasa)
• Fix agent UI redirect loop during cloud sign-in #6868 (jacekkolasa)
• Fix /var/lib/netdata/registry getting left behind after uninstall #6867 (davent)
• Fix python.d.plugin bug in parsing configuration files with no explicitly defined jobs #6856 (ilyam8)
• Fix potential buffer overflow in the web server #6817 (amoss)
• Fix netdata group deletion on linux for uninstall script #6645 (mbarper)
• Various cppcheck fixes #6386 (ac000)
• Fix crash on FreeBSD due to do_dev_cpu_temperature stack corruption #7014 (samm-git)
• Fix handling of illegal metric timestamps in database engine #7008 (mfundul)
• Fix a resource leak #7007 (vlvkobal)
• Fix rabbitmq collector error when no vhosts are available. #7018 (mfundul)