The open-source observability platform everyone needs!
GPL-3.0 License
Bot releases are visible (Hide)
Netdata v1.45.5 is a patch release to address issues discovered since v1.45.4.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 6 months ago
Netdata v1.45.4 is a patch release to address issues discovered since v1.45.3.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 6 months ago
[!WARNING]
Important Security UpdateNetdata v1.45.3 is a patch release to fix a local privilege escalation vulnerability discovered in v1.45.x releases. Users are advised to upgrade any systems running v1.45.0, v1.45.1, or v1.45.2 immediately. Stable releases before v1.45.0 are unaffected by this vulnerability. Full details on the vulnerability can be found in the associated security advisory on GitHub. A big thank you to mia-0 for identifying and reporting this issue!
This patch release also addresses other issues discovered since v1.45.2.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 7 months ago
Netdata v1.45.2 is a patch release to address issues discovered since v1.45.1.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 7 months ago
Netdata v1.45.1 is a patch release to address issues discovered since v1.45.0.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 7 months ago
Thanks to your love ❤️, Netdata is leading the observability category in CNCF, having significantly more stars than Elasticsearch, Grafana, Prometheus and all other observability solutions listed in CNCF landscape.
We are committed to provide the most advanced and innovative observability solution, to help us minimize monitoring costs while providing AI-powered high-fidelity monitoring!
You like Netdata? Give Netdata a ⭐ too, on GitHub!
3 months have passed since the previous Netdata release. A lot has changed since then! Netdata now has a mobile app for alert notifications, new drag-and-drop custom dashboards, network connections monitoring, dynamic configuration for data collection jobs and alerts, and many more...
To see how Netdata stacks up against the most advanced commercial offerings available today, we did an analysis on how Dynatrace, Datadog, Instana, Grafana and Netdata commercial offerings compare.
It is nice to see that Netdata stands out for its:
Excellent technology coverage
Netdata's monitoring coverage is significantly higher compared to others, in all areas!
High-fidelity, real-time insights
Netdata is the only monitoring solution offering this kind of fidelity (per-second for all metrics), at this extend!
Real AI and Machine-Learning.
Netdata is the only monitoring system that offers real machine learning, running at the edge.
Lightweight
Netdata is among the lightest agents, despite the fact that it does a lot more than the others.
Best cost efficiency
Netdata's cost efficiency is unbeatable, making Netdata the most cost-efficient monitoring solution available today!
You can now receive Netdata alerts directly on your mobile phone!
Choose your space and see all the available notifications since you last signed in!
Check the full demo here.
The Mobile App is available for Homelab and Business plan users.
You can now create advanced custom dashboards with Netdata!
Drag-and-Drop
Easily move charts from Metrics or Single Node views straight to your dashboards. It's intuitive and fun.
New Chart Types
Discover your data in new ways with Bar, Circle, Gauge, Pie, Value, and Group boxes.
Quick Dashboard Creation
Hit the plus button, drag, and you've got a new dashboard. Simple as that.
Rename Charts
Customize your dashboard by renaming charts to whatever makes sense to you.
Refreshed Text Cards
We've upgraded text cards for better clarity and aesthetics.
On the Agent UI and the Community plan of Netdata Cloud 1 custom dashboard is allowed. The Homelab and Business plans of Netdata Cloud support an unlimited number of custom dashboards.
Explore the network connections of your servers and processes!
Netdata got a network viewer (select network-connections
from the Top
tab inside the dashboard).
The tool reports all IPv4 and IPv6, TCP and UDP sockets a system and all its processes have. Also, it automatically and reliably classifies them as inbound
, outbound
, local
(i.e. within the host itself), or listen
(for daemons).
The visualization graph has 4 sides:
public
(i.e. public IPs),private
(i.e. private and reserved IPs),servers
(i.e. listening and inbound sockets),clients
(i.e. sockets towards other servers).The position of each application on the chart is determined by the classification of the sockets it has. To the top are clients, to the bottom are servers, to the right are internet facing applications, to the left is internal network applications.
The size of each application in the chart is determined by the number of sockets it has, and each application is a pie chart representing the percentage of each kind of sockets it has.
For servers with dozens of thousands of sockets, the tool provides an aggregated view, grouping similar sockets together and reporting the total. Users can switch to a detailed view from the UI.
We've improved immensely the customization capabilities of Netdata with the introduction of User Settings.
Our first release on this front is focused on the customization of charts, either on the Metrics tab or the Single Node view tab. You can now create for any chart:
With this, you can define what is best for your team to visualise a given chart but still allow each teammate to define their own. Users will be presented with the view they should see, based on setting hierarchy, but they
can interchangeably select which of the select views they want.
More areas of customization will come soon, Filters saved views, Dashboards Table of Content (TOC) ordering, etc.
Netdata agents are now deployed with the ability to dynamically accept configuration from the UI, for data collection jobs and alerts. The feature is released in beta.
Check the full demo here
🔕Improvements done to make it easier to interact and see Alert Silencing Rules.
With this release, you will be able to:
We hope this makes it easier for you to interact with the Alert Silencing Rule Manager.
Stay tuned for more improvements!
Netdata's apps.plugin
has been ported to macOS, allowing users to view processes information on Linux, FreeBSD and macOS!
Just install the latest Netdata on your macOS and enjoy full processes monitoring!
For non-professional use, get the whole and the latest of Netdata! For the cost of a beer per month, you can get access to all Business features of Netdata, for your home lab or personal project!
Our Homelab plan is available to technology enthusiasts and students, for non-professional user, offering the entire Netdata suite, for a small flat fee, under a fair usage policy.
Unlimited Access: Enjoy the freedom of unlimited usage, with no caps on nodes or custom dashboards.
Premium Features: Get your hands on business-level features, including enhanced alert integrations and access to our mobile app, all tailored for your personal projects.
Support Netdata: Support the open-source Netdata, to ensure it will be there for you, when you need it!
Starting with Netdata 1.45, we have completely removed our GNU Autotools based build system and replaced it with
CMake. The new CMake build system has a number of significant benefits for developers, package maintainers, and
those using local builds of Netdata.
-G Ninja
option to CMake during the configuration process.Most users should not be directly affected by this change other than benefiting from the faster build times,
only those who were building locally by hand (not using the netdata-installer.sh
script or the kickstart script)
will need to change things.
Alongside the new CMake build system, we have also moved the go.d.plugin
code from the netdata/go.d.plugin repository to the main netdata/netdata
repository.
We have made this change for three reasons:
Users of native packages and static builds should see no difference at all from this change.
Building the agent locally will now require a working Go toolchain supporting a particular minimum version of the
Go language (currently 1.21) if the Go plugin needs to be built. The plugin itself can still be disabled to avoid
this requirement, but this is not recommended.
The installer code will attempt to ensure that a sufficiently up-to-date Go toolchain is installed when installing
or updating the agent. If such a toolchain is not found, it will attempt to automatically install a copy of the
official toolchain from https://go.dev/dl/ in /usr/local/go
. If that attempt fails, the Go plugin will be
DISABLED automatically at build time.
build
(#16768, @Ferroin)All depreciated items from the v1.44.0 notice have been addressed except for enabling gorilla compression by default.
Additionally, the following Alert options have been deprecated in this release. While Netdata will still understand these options when
reading existing alert configurations for now, we recommend updating your custom alert configurations to use the
replacements listed below. Compatibility with these deprecated options might be removed in a future release.
Option | Use instead |
---|---|
foreach DIMENSIONS (lookup line) |
- |
charts |
- |
os |
host labels: _os=X |
host |
host labels: _hostname=X |
plugin |
chart labels: _collect_plugin=X |
module |
chart labels: _collect_module=X |
Where X is a simple pattern.
Join the Netdata team on the 25th of March at 17:00 UTC for the Netdata Release Meetup.
Together we’ll cover:
RSVP now - we look forward to meeting you.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 8 months ago
Netdata v1.44.3 is a patch release to address issues discovered since v1.44.2.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 9 months ago
Netdata v1.44.2 is a patch release to address issues discovered since v1.44.1.
This patch release provides the following bug fixes and updates:
We would like to thank our dedicated, talented contributors who make up this amazing community. The time and expertise that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a remarkable product.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 11 months ago
Netdata v1.44.1 is a patch release to address issues discovered since v1.44.0.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 11 months ago
Steady to our schedule, this is another great Netdata release!
[!IMPORTANT]
Stay informed about upcoming changes and potential deprecations by reviewing the deprecation notice sections. This will help you plan for any necessary adjustments to ensure a smooth transition.
66k+ GitHub Stars ⭐
Since October 2023, Netdata is leading the observability category in the CNCF landscape, surpassing Elasticsearch. Thank you for your love ❤️! Give Netdata a ⭐ too, on GitHub!
600M+ docker hub pulls
Netdata runs with about 200k docker hub downloads per day. Since June 2023 we are a Verified Publisher, so that Netdata pulls don't count against docker hub pull limits for our users, allowing all our users to integrate Netdata to their CI/CD toolchains.
log2journal
: a utility to extract, convert, transform and send to systemd-journal any kind of structured logs (including JSON and logfmt logs), similar to what promtail
does for Loki.We tested Netdata and Prometheus at scale, both ingesting 2.7 million metrics per second. On the same workload, Netdata vs Prometheus needs:
Read the full performance comparison between Netdata and Prometheus.
To achieve these astonishing results, we made the following changes to Netdata since the previous release:
SLOTS
streaming protocolA new streaming protocol, allows Netdata children and parents to share a common index of the metrics streamed, allowing the parents to receive metrics without consulting hashtables, reducing the overall overhead on parents by about 30%, without increasing the overhead on children (the children just number each metric).
The new protocol, called SLOTS
, is automatically selected when both the child and the parent support it.
Streaming now supports multiple compression algorithms. Previous Netdata releases supported only LZ4, which is known for its speed and average compression ratio. This release adds support for ZSTD, GZIP, and BROTLI.
ZSTD provides the best balance between compression ratio and CPU consumption, and therefore it is now the default.
The compression algorithms selection order can be configured on parents, in stream.conf
, at the [API]
section (parents), by setting compression algorithms order = zstd lz4 brotli gzip
.
If you need to save most bandwidth at the expense of CPU utilization set this so that brotli
or gzip
appear first in the list, before zstd
and lz4
.
This also means that parents can now have a different compression order for each API key, allowing the use of different API keys based on the location of the child (i.e. children that are on billable egress bandwidth can use an API key that prefers the best compression, like brotli
and gzip
, while children on non-billable egress bandwidth can use an API key that prefers the best CPU utilization, like zstd
or lz4
).
Gorilla compression is a time series data compression technique, developed by Facebook for their time series database, Gorilla. It's particularly efficient for compressing data that changes incrementally over time, which is a common characteristic of time series data.
This release of Netdata includes an adaptation of Gorilla compression, which once enabled, provides 30% additional memory reduction to Netdata.
This was not ready when we compared Netdata and Prometheus, so the Gorilla compression benefits weren't accounted in the comparison. By enabling Gorilla compression, Netdata memory reduction is 70%+ compared to Prometheus.
To try Gorilla compression, edit netdata.conf
and set at the [db]
section, dbengine page type = gorilla
.
Keep in mind that enabling Gorilla compression changes the dbegnine file format to Gorilla compressed metrics. This version of Netdata can read Gorilla-compressed data from dbengine even if Gorilla compression is not enabled, but previous versions of Netdata cannot read it. So, enable Gorilla, only if you don't plan to switch back to a previous version of Netdata.
Our plan is to have Gorilla compression enabled by default at the next release of Netdata.
Our systemd-journal.plugin
was already quite faster (10x) than journalctl
, but still it was slow when the journal databases is huge (e.g. at journals centralization points where hundreds or thousands of nodes push their logs).
In this release, we introduce several changes to allow the plugin to work promptly in such environments.
The biggest performance issue with systemd-journal logs is the query performance when dealing with huge logs databases.
To overcome this performance issue and provide prompt responses to queries, Netdata now uses the following strategy:
[unsampled]
at the histogram. We know these log entries are there, but the value counters on the field filters do not include them.[unsampled]
threshold is hit, and we have read more than 1% of each file, Netdata estimates the number of entries that will be read from the file and skips the rest of it. This estimation appears as [estimated]
in the histogram.The above process allows Netdata to provide a histogram of the logs in a timely manner, even when the number of log entries in the visible timeframe is several dozen million.
A similar process is usually used by log management systems, including Grafana Loki and Elasticsearch. However, Netdata takes a much bigger sample of the data (other systems usually sample only a few thousand log entries, while Netdata usually samples more than a million) and the visualization allows exposing the exact sampling and estimations made at the histogram.
Image showing [unsampled]
and [estimated]
on a systemd journal system that collects about 10k nginx log entries per second:
Read more about journals query performance.
On busy logs centralization servers, the number of journal files available in /var/log/journal/remote
can grow significantly, slowing down directory listing (even ls -l
is very slow on them).
To overcome this issue, Netdata now uses inotify events and sorts the files to be scanned from the latest to the oldest.
These changes allow Netdata to present the logs user interface for the most recent journals, immediately after a Netdata restart, while the journals database is scanned in the background.
We switched Netdata docker images from Alpine Linux to Debian, so that libsystemd
will be available inside the docker image, allowing systemd-journal.plugin
to be compiled and shipped with Netdata docker images.
Using Netdata docker images, Netdata can now query the host system journal files, while running inside the container.
systemd-journal has a nice feature where certain events of common interest are given a specific MESSAGE_ID
. Several such MESSAGE_ID
s have been assigned to track common events, like coredumps, units start/stop events, VMs start/stop events, time changes, etc. In total, we found more than 50 total unique events that are tracked this way.
This version if systemd-journal.plugin
automatically tracks and annotates these MESSAGE_ID
s using their names allowing quick spotting of events of common interest.
This feature is available at the MESSAGE_ID
field filter, at the right side of the dashboard.
log2journal
, a new tool on your quiver for managing logs
log2journal
is a new utility allowing the conversion of log files into structured systemd-journal log entries. This is currently in beta.
The utility allows processing logs like this:
tail -F /var/log/nginx/access.log |\
log2journal -c nginx-combined |\
systemd-cat-native
The above builds a basic pipeline for converting the access.log
of an Nginx web server into structured log entries in the local systemd-journal.
tail
is responsible for feeding the latest logs lines to log2journal
. Multiple files can be specified and log2journal
can also pick up the filename from tail
and add it as a field to the journal logs.log2journal
extracts fields from the log lines it is fed with. This is a powerful tool that can read json
and logfmt
logs, but also extract fields using PCRE2 patterns from any log. It supports filtering, renaming, and rewriting rules using command line arguments or yaml configuration files. The output of log2journal
is the standard Journal Export Format.systemd-cat-native
is another new Netdata utility, reading standard Journal Export Format entries, which are then sent to a local or remote systemd-journal system.Image showing structured nginx logs into systemd-journal:
The logging layer of Netdata has been rewritten, so that Netdata logs now go to the systemd-journal, in a namespace called netdata
.
The obvious outcome is that now you can monitor Netdata logs, using Netdata's systemd-journal.plugin
user interface and thanks to journal namespaces, this does not pollute the system logs. But this is just the beginning...
Netdata utilizes the MESSAGE_ID
feature of systemd-journal to register:
This means that the systemd-journal.plugin
user interface, and journalctl
can now be used to list all such events uniformly.
Screenshot of Netdata alert transitions in systemd-journals:
All Netdata logs are now structured. Netdata can also log in json
or logfmt
formats. We introduced a lot of new fields to track every aspect of Netdata, in a uniform and consistent way. Read more here.
Furthermore, we introduced a new tool called systemd-cat-native
allowing any application or shell script to send structured logs to systemd-journal. Read more here.
Several new Functions have been added to help us in our troubleshooting journeys. On top of processes
, streaming
and systemd-journal
, we are leveraging the wide range of collectors and metrics Netdata has and bring data in a different visual representation.
The updated list can be found on our documentation here, and you can find a summary of the currently available functions with the corresponding CLI tool it relates to:
Function | Description | Alternative to CLI tools | plugin - module |
---|---|---|---|
block-devices | Disk I/O activity for all block devices, offering insights into both data transfer volume and operation performance. | iostat |
proc |
containers-vms | Insights into the resource utilization of containers and QEMU virtual machines: CPU usage, memory consumption, disk I/O, and network traffic. |
docker stats , systemd-cgtop
|
cgroups |
ipmi-sensors | Readings and status of IPMI sensors. | ipmi-sensors |
freeipmi |
mount-points | Disk usage for each mount point, including used and available space, both in terms of percentage and actual bytes, as well as used and available inode counts. | df |
diskspace |
network interfaces | Network traffic, packet drop rates, interface states, MTU, speed, and duplex mode for all network interfaces. |
bmon , bwm-ng
|
proc |
processes | Real-time information about the system's resource usage, including CPU utilization, memory consumption, and disk IO for every running process. |
top , htop
|
apps |
systemd-journal | Viewing, exploring and analyzing systemd journal logs. | journalctl |
systemd-journal |
systemd-list-units | Information about all systemd units, including their active state, description, whether or not they are enabled, and more. | systemctl list-units |
systemd-journal |
systemd-services | System resource utilization for all running systemd services: CPU, memory, and disk IO. | systemd-cgtop |
cgroups |
streaming | Comprehensive overview of all Netdata children instances, offering detailed information about their status, replication completion time, and many more. |
In the short-term, we will keep adding more (hopefully) helpful Functions but have longer-term plan where we will want to expand this functionality to potentially allow taking and storing snapshots of the results based on: triggered alerts, or periodical configuration.
In case you have suggestions we have a running GitHub Discussion open here.
We've been working on adding more Alert Notification Integrations to Netdata Cloud and recently added the following new ones:
The full list of Alert Notification Integrations from Netdata Cloud can be found on our documentation here.
cgroup.procs
/tasks
if it does not exist (cgroups.plugin) (#16196, @ilyam8)systemd-cat-native
(#16456, @ilyam8)cpuset.cpus
(#16385, @ilyam8)title
in /api/v1/charts (#16416, @ilyam8)In accordance with our previous deprecation notice, the following items in this release have been changed:
Other unannounced changes:
Netdata internal metrics (Netdata Monitoring section) are disabled by default to reduce the overall data volume. Later we plan to enable only important internal metrics by default.
Can be enabled in netdata.conf
by uncommenting and changing no
to yes
:
[plugins]
# netdata monitoring = no
# netdata monitoring extended = no
Logging
netdata.conf
file and make the necessary changes under the [logs]
section.To ensure seamless compatibility with future updates, we recommend transitioning from source-built installations to our distribution packages or static binaries. Starting with our next release, we will no longer guarantee compatibility when updating source-built installations. This change allows us to focus on enhancing the stability and feature delivery for the rest of our supported installation methods.
Gorilla compression will be enabled by default.
The Google Cloud Pub Sub and the AWS Kinesis exporters will be removed in the next release. Both of them were not maintained and were not used when building packages. Users can consult the exporting documentation for alternative exporters to use.
The database modes map
and save
will be removed in the next release. The dbengine
database mode will be used to persist metrics on disk automatically.
Per-core CPU metrics will be disabled by default to reduce data volume. Summary (per-system) metrics are still collected. This change enhances performance and resource utilization. Disabled metrics:
cpu.cpu
(utilization).cpu.interrupts
(all interrupts).cpu.softirqs
(software interrupts).cpu.softnet_stat
(software interrupts related to network receive work).cpu.cpu_cstate_residency_time
(idle states).Can be enabled in netdata.conf
by uncommenting and changing no
to yes
:
[plugin:proc:/proc/stat]
# per cpu core utilization = no
# cpu idle states = no
[plugin:proc:/proc/interrupts]
# interrupts per core = no
[plugin:proc:/proc/softirqs]
# interrupts per core = no
[plugin:proc:/proc/net/softnet_stat]
# softnet_stat per core = no
To optimize system performance, several eBPF.plugin modules have been disabled by default. While these modules provide valuable insights into system resource usage, they can also contribute to system overhead. They will expose metrics using Functions (run on demand and for a limited period of time). These modules include:
Join the Netdata team on the 11th of December at 16:30 UTC for the Netdata Release Meetup.
Together we’ll cover:
RSVP now - we look forward to meeting you.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 12 months ago
Netdata v1.43.2 is a patch release to address issues discovered since v1.43.1.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot 12 months ago
Netdata v1.43.1 is a patch release to address issues discovered since v1.43.0.
This patch release provides the following bug fixes and updates:
We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise
that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a
remarkable product.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot about 1 year ago
systemd-journal
logs release!Steady to our schedule, this is another great Netdata release!
65.5 k GitHub Stars ⭐
Since October 2023, Netdata is leading the observability category in the CNCF landscape, surpassing Elasticsearch. Thank you for your love ❤️! Give Netdata a ⭐ too, on GitHub!
595 M docker hub pulls
Netdata runs with about 200k docker hub downloads per day. Since June 2023 we are a Verified Publisher, so that Netdata pulls don't count against docker hub pull limits for our users, allowing all our users to integrate Netdata to their CI/CD toolchains.
This release is the most robust and reliable Netdata we have ever built.
These are the main areas Netdata has improved since the last release:
Logs
Today we release an almost rewritten version of systemd-journal
, to improve its performance and visualization capabilities. systemd-journal
holds critical systems and security information and given the lack of systemd-journal
visualization tools, we focused first on filling this gap. At the same time, we are standardizing the way logs should be as a part of Netdata, enabling us to support more log management engines, like Loki and Elasticsearch.
Instances Slice and Dice
Given the capabilities of the new Netdata Agent UI (v2), we are changing the way some of our collectors collect and expose metrics, to allow easier slicing and dicing of the data and be more OpenTelemetry compatible in terms of specifications. So, in this release we changed the way apps.plugin
exposes charts in the Applications
section of the dashboard. Following the NIDL framework, each application group is now an instance, allowing better aggregation of processes utilization across nodes. Similarly, our systemd
units charts have been updated to have an instance for each systemd
unit. For the same reasons, disk charts now have additional labels (id
, model
and serial
) to help us identify disks from the charts. Unfortunately, such changes tend to make the older dashboards (v1, v0) less usable, especially on servers with many hundreds of instances.
Stock Alerts
A number of changes have been implemented to the Netdata Health engine, to allow better integration with the new dashboard. More changes in this area are about to come, as part of the next release: a) allow multi-node alerts on parents, b) allow evaluating and configuring alerts from the UI.
Alerts Accuracy
Netdata has by default 3 tiers of metrics, each with a different resolution. The Netdata query planner is automatically picking the right tier to satisfy a query, based on the number of points requested in the response. For alerts there was a side effect. Since alerts request only 1 point of data in the response, the query planner was picking the "easier" tier to query, which is of course the one with the lower resolution. Now alerts are always run on tier 0, the higher resolution one.
Lower Resources Utilization
Several changes have been implemented for Netdata to better take care of itself. That includes lower memory usage, lower disk footprint, self vacuuming of SQLite databases, and more. Probably the most notable change is that now Netdata needs only 1 pointer (8 bytes on 64 bit, 4 bytes on 32 bit) for each use of a label name-value
combination. This improves drastically Netdata's memory requirements in setups like busy k8s clusters, that containers come and go all the time, increasing the labels cardinality significantly.
32bit Netdata on 64bit IoT machines
A common request when Netdata is installed on 64bit IoT devices, is to have a 32bit Netdata running there. Before this release, this was not possible. Now a 32bit Netdata will nicely run on a 64bit operating system.
Netdata Cloud on prem
Netdata Cloud is now available to be installed on-prem! Several companies have already deployed it and are currently testing it. If you want to join them, submit this form.
systemd-journal
systemd-journal
was first included in Netdata v1.42.0. Immediately after release, we recognized the wider need for this feature, so we've rewritten the plugin almost entirely, to provide the best possible experience. This work is also fundamental for supporting more log monitoring integrations - stay tuned!
The major improvements done on systemd-journal
logs function were:
journalctl -f
, showing new logs entries immediately after they are receivedjournalctl
doesIf you want to take a look at a full presentation of the systemd-journal
plugin, how it works, how you can take full advantage of this and even instructions on configuration of a logs centralization server, check the documentation for the plugin.
You can experience the power of systemd-journal
logs function in one of our Netdata demo rooms here
or check our latest YouTube video on it.
Want to know why you should untap the full potential of systemd-journal
logs? Check out Netdata's founder, Costa Tsaousis @ktsaou, blogpost on it here.
With the increased feedback and requests on VMware vCenter Server collectors we have:
host
, datacenter
, cluster
, vm
It is with this feedback from the Community that we can keep working on improving Netdata to ensure it meets
your needs!
We are currently working on the following areas, which we hope to release next month:
Logs Explorer for Loki and Elasticsearch
Similar to systemd-journal
, allow Netdata to explore, query and visualize logs from Loki and Elasticsearch.
Collectors Configuration from the UI
In the last release we presented the Integrations Marketplace. Since then, we work to make all integrations configurable via the dashboard. This will allow all of us to configure our Netdata servers directly from the UI, without touching configuration files, improving significantly the usability and easiness of Netdata.
Alerts Configuration from the UI
Similarly, we work to allow configuring alerts directly from the UI, without text file configurations, so the all of us can create powerful alerts on the spot.
Netdata Mobile App
We are at the final stage of releasing our Netdata Mobile App (iOS and Android) for receiving mobile push notifications and exploring alerts statuses.
Scalability
Given the wide adoption of Netdata, we are committed to make Netdata scale better in larger environments. Especially when it comes to Netdata parents, we aim to provide the best scalability possible. We are currently finalizing the necessary changes to allow Netdata achieve:
Of course, the numbers depend on the CPU and its clock, but they shouldn't vary significantly on modern systems.
At the same time, we work to integrate Gorilla compression to our database. This will provide a significantly better overall memory footprint for Netdata.
We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a remarkable product.
netdata/ansible
playbookdelete old models param
to ML readme (#15873, @andrewm4894)netdata/ansible
(netdata/ansible#6, @luisj1983
anomaly_detection.detector_events
chart (#16028, @andrewm4894)anomaly_detection.type_anomaly_rate
stacked (#15895, @andrewm4894)In accordance with our previous deprecation notice, the following items in this release have been changed:
Component | Type | Change | Action |
---|---|---|---|
apps.plugin | collector | a dimension for each group/user/user group => a chart for each group/user/user group | |
cgroups.plugin | collector | a dimension for each systemd service => a chart for each systemd service | |
proc.plugin | collector | all "Networking Stack" metrics except "tcp" have been moved to "IPv4 Networking" | |
family attribute |
alert configuration and Health API | deprecated | use chart labels |
We plan to change in the next release (v1.44.0):
Component | Type | Change | Action |
---|---|---|---|
charts.d/nut | collector | deprecated | use go.d/upsd |
Join the Netdata team on the 18th of October at 16:30 UTC for the Netdata Release Meetup.
Together we’ll cover:
RSVP now - we look forward to meeting you.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot about 1 year ago
Netdata v1.42.4 is a patch release to address issues discovered since v1.42.3.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot about 1 year ago
Netdata v1.42.3 is a patch release to address issues discovered since v1.42.2.
This patch release provides the following bug fixes and updates:
We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise
that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a
remarkable product.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot about 1 year ago
Netdata v1.42.2 is a patch release to address issues discovered since v1.42.1.
This patch release provides the following bug fixes and updates:
We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise
that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a
remarkable product.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot about 1 year ago
Netdata v1.42.1 is a patch release to address issues discovered since v1.42.0.
This patch release provides the following bug fixes and updates:
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot about 1 year ago
Steady to our schedule, this is another great Netdata release!
64.5 k GitHub Stars ⭐
Netdata got at the top trending repos on GitHub, after the last release. ❤️ Thank you for your love! 🚀 You rock!
580+ M docker hub pulls, running at 200+ k per day.
Netdata is a verified publisher on Docker Hub, and our users enjoy free unlimited Docker Hub pulls!
A beta version of the Netdata Marketplace is included in this release:
More than 800 integrations are available, directly from the dashboard. For each integration, all the information required to get it up and running is included:
Integrations are still in beta. We improve it every day, but we think it is already quite useful.
A new Netdata Function has been added to query the systemd journal logs:
The function respects the current date-time picker, so it can query any possible timeframe the systemd journal has data for.
IMPORTANT
Netdata Functions are available only when you are signed in to Netdata and your Netdata Agent is claimed.
This has been done to protect your privacy. Netdata Cloud checks that the users of the Agent dashboard are allowed to view this information.
IMPORTANT
Thesystemd-journal
function is currently available only on Netdata Agents that have been installed from source, or with native packages of the Linux distribution (RPM, DEB). For users running static builds of Netdata or running Netdata in a Docker container, we are working to bringsystemd-journal
to them too. Stay tuned...
You can now connect your agents to Netdata Cloud, via the dashboard:
The UI verifies that you are the owner of a Netdata, by asking you to provide a random key that is saved to a file on disk. Once you provide the right key, Netdata is automatically claimed to your space at Netdata Cloud.
The UI has an AR
button above the menu. When you press it, the dashboard queries the Netdata Metrics Scoring Engine, to find the anomaly rates for the visible timeframe, across the metrics included in the dashboard. Then it add a badge next to each category and subcategory, showing its anomaly rate.
This way, you can quickly spot what is anomalous on the current view of the dashboard.
We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a remarkable product.
diskquota
collector to third party collectors list (#15524, @andrewm4894)We plan to change the following items in the next release (v1.43.0):
Component | Type | Change | Action |
---|---|---|---|
apps.plugin | collector | a dimension for each group/user/user group => a chart for each group/user/user group | |
cgroups.plugin | collector | a dimension for each systemd service => a chart for each systemd service | |
proc.plugin | collector | all "Networking Stack" metrics except "tcp" => "IPv4 Networking" | |
python.d/nvidia_smi | collector | deprecated | use go.d/nvidia_smi |
family attribute |
alert configuration and Health API | deprecated | use chart labels |
Join the Netdata team on the 11th of August at 17:00 UTC for the Netdata Release Meetup.
Together we’ll cover:
RSVP now - we look forward to meeting you.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot over 1 year ago
Checkout the v1.41 release meetup recording or read on to learn more about the new UI and other features in this release.
Steady to our schedule, this is another great Netdata release!
❤️ Thank you for your love! 🚀 You rock!
Netdata Agents and Parents now have a new UI!
New CHARTS 🟢 New SUMMARIES 🟢 MACHINE-LEARNING FIRST 🟢 INFRASTRUCTURE LEVEL DASHBOARDS 🟢 FILTER, SLICE, and DICE any dataset 🟢 ANOMALY ADVISOR 🟢 METRICS CORRELATIONS 🟢 NETDATA FUNCTIONS 🟢 EVENTS FEED 🟢 HEATMAPS 🟢
In the last few months, we have ported and open-sourced all Netdata Cloud APIs to the Netdata Agent, allowing Netdata Parents to drive the same multi-node / infrastructure level dashboards Netdata Cloud provides!
So, as of today, Netdata Agents and Parents present the same UI, exactly the same dashboard, charts and features with Netdata Cloud!
Apart from the entirely new look, single-node dashboards now group similar charts together. So, all disk drives, network interfaces, cgroups (containers and VMs), are now a single set of charts.
This allows Netdata to aggregate a vast amount of datasets in a chart, like the following, where almost 20k containers are now manageable:
To make it easier for you to navigate, filter, slice, and dice the data, the menus above each chart give you easy access to all the data of the chart:
When Netdata Agents are configured as Parents (multiple other agents stream metrics to them), they now present multi-node and multi-instance charts. At the top right corner of the dashboard, there is the global nodes filter, from which you can slice the entire dashboard for one or a few of your nodes.
Get a firsthand walkthrough with Costa Tsaousis, Netdata's Founder, on the rationale for this change and the path Netdata is taking by checking the video from Netdata Office Hours on YouTube.
You can still access all versions of the dashboards, as follows:
http://your.server:19999/
The default dashboard is now a live version of the new UI. The dashboard static files are served by Cloudflare and are automatically updated when we release a new version of the UI, so that your Netdata agent is always up to date.
http://your.server:19999/v2/
A local copy of the latest dashboard, as it was at the time the agent was released. This is distributed with Netdata under the Netdata Cloud UI License v1.0. The local copy is automatically used if for any reason the web browser cannot download the live version of it.
http://your.server:19999/v1/
The previous single-node version of the Netdata Agent dashboard.
http://your.server:19999/v0/
The now ancient, original version of the Netdata Agent dashboard.
Netdata Assistant: Your AI-Powered Troubleshooting Sidekick
The Netdata Assistant is an AI-powered tool that uses large language models and our community's knowledge to guide you during troubleshooting and help you get to the root cause sooner.
The goal of the Netdata Assistant is straightforward: to make your troubleshooting process easier. It's here to save you from the hassle of sifting through tons of information so you can focus on solving the problem at hand.
It will give you the lowdown on the alert, why it's happening, and why you should care. It'll also guide you on how to troubleshoot it and even offer some handy web links for more info if you're interested.
Read more about it on the Netdata blog here.
Netdata got a new FreeIPMI collector. The new collector is able to collect IPMI sensors at a much better data collection rate, and it is more reliable and robust compared to the previous one.
We have also categorized all sensors based on the component they monitor:
And provided as labels the exact sensor name each metric refers to:
"FD" stands for "file descriptor". A file descriptor is an integer that the operating system assigns to an open file to track it. This includes regular data files, directories, network sockets, pipes, and other types of I/O streams.
In Linux, everything is treated as a file, which includes hardware devices, directories, and sockets. Each open file is assigned a file descriptor. When a file is closed, its file descriptor is freed up for reuse. However, if an application doesn't close a file when it's done with it, that's called a "file descriptor leak".
File descriptor leaks can cause several problems:
Resource exhaustion: Each process has a limit to the number of file descriptors it can open. If a process continually leaks file descriptors without closing them, it will eventually hit this limit and won't be able to open any more files, which often causes the process to crash.
Unexpected behavior: Open file descriptors hold resources, like network sockets, that might be expected to be available for other uses. If these resources are tied up due to a leak, it can cause unexpected behavior.
Security issues: File descriptors can sometimes be used to gain unauthorized access to data if they're not properly managed.
apps.plugins
is now able to track the usage of FDs against the limits set for each application. We have added an fds
category in the Applications
section of the dashboard. The first chart shows the percentage of FDs used by each application against its limits:
We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a remarkable product.
info
macro with a less generic name.nan
(#15348, @ilyam8)error
function (#15296, @thiagoftsm)info
macro with a less generic name (#15266, @carlocab)There is not an obvious list of items that will be deprecated in the upcoming release (v1.42.0). Feel free to check and elaborate on the upcoming backlog
In accordance with our previous deprecation notice, the following items in this release:
Component | Type | Will be replaced by |
---|---|---|
python.d/nvidia_smi | collector | go.d/nvidia_smi |
family attribute |
alert configuration and Health API | chart labels attribute (more details on netdata#15030) |
Join the Netdata team on the 21st of July at 17:00 UTC for the Netdata Release Meetup.
Together we’ll cover:
RSVP now - we look forward to meeting you.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:
Published by netdatabot over 1 year ago
Netdata v1.40.1 is a patch release to address issues discovered since v1.40.0.
This patch release provides the following bug fixes:
We would like to thank our dedicated, talented contributors that make up this amazing community. The time and expertise
that you volunteer are essential to our success. We thank you and look forward to continuing to grow together to build a
remarkable product.
As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter
an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us
through one of the following channels: