PostgreSQL Cluster performances monitoring and auditing tool
POSTGRESQL License
Bot releases are hidden (Show)
2024-03-06 - Version 4.0
This release is a major release that fixes issues reported since
the past six months and adds several new features.
New features:
CREATE TABLE my_client_pgstats (
export_time TEXT,
dbname TEXT,
schemaname TEXT,
tablename TEXT,
attname TEXT,
inherited BOOLEAN,
null_frac REAL,
avg_width INT,
n_distinct REAL,
most_common_vals TEXT[],
most_common_freqs REAL[],
histogram_bounds TEXT[],
correlation REAL
);
Note that the columns most_common_vals and histogram_bounds have
the type TEXT[] instead of anyarray.
Compatibility:
There is not compatibility issues but a change of behavior with default
to dump the pg_stats view. Following the number of table/indexes in the
databases it could take more time collecting the statistic so in this case
the interval of polling should be increase. There is an new option to disable
the pg_stats dump, --no-pg_stats-dump, if you want to get back to the previous
behavior.
Here is the full list of fixes since previous release.
Published by darold over 1 year ago
2023-07-11 - Version 3.5
This is a maintenance release that fixes issues reported since
the past six months and adds support to PostgreSQL 16.
Published by darold almost 2 years ago
2022-12-26 - Version 3.4
This release is a maintenance release that fixes issues reported since
the past six months.
Published by darold over 2 years ago
2022-06-02 - Version 3.3
This release is a maintenance release that fixes issues reported since previous release. It also adds new reports and features.
- Add TCP utilization and TCP errors reports, from sar -nTCP,ETCP.
- Add a comment in the suggested index if it could be redundant
with existing ones.
- Add support to run pgcluu_collect as user with the pg_monitor role.
Here is the full list of changes since previous release.
- Add support to run pgcluu_collect as user with the pg_monitor role.
Thanks to Nicolas Gollet for the report.
- Fix redundant index query to work with PG < 10, regexp_match() must be
replaced by regexp_matches() with some additional rewrite.
- Replace call to pg_ls_dir by pg_ls_waldir to be able to use pg_monitor
privilege starting from PG10. Thanks to Nicolas Gollet for the report.
- Fix impossibility to use --lock-timeout. Thanks to Christophe Courtois
for the report.
- Remove lock_timeout from internally set of PGOPTIONS when querying
pgbouncer, the old value is restored after. Thanks to Christophe Courtois
for the report.
- Fix empty pgbouncer statistics file when running in interactive mode.
Thanks to Christophe Courtois for the report.
- Add a comment in the CREATE INDEX when the suggested index may be redundant
with existing ones. Thanks to Christophe Courtois for the patch.
- Fix false report of redundant index when the WHERE clause is different.
Thanks to Guillaume Lelarge for the report.
- Fix in pgcluu for OUTPUT_DIR always be quoted, especially if their value
may contain a white space or separator character. Thanks to Nicolas Gollet
for the patch.
- Fix network errors disabled. Thanks to Christophe Courtois for the patch.
- Add TCP utilization and TCP errors reports, from sar -nTCP,ETCP. Thanks to
Christophe Courtois for the patch.
- Fix pgbouncer command. Thanks to Nicolas Gollet for the patch.
- Modification of the query on pg_stat_statement so that under PostgreSQL 13
and higher total_time = total_plan_time + total_execution_time. Thanks to
Nicolas Gollet for the patch.
- Fix STATS_REPORT_RETENTION directory in config file. Don't prepend DESTDIR
to paths in config file. Thanks to Christoph Berg for the patch.
Published by darold about 3 years ago
2021-09-30 - Version 3.2
This release is a maintenance release that fixes issues reported since previous release. It also adds new reports and features.
Backward compatibility:
Here is the full list of changes since previous release.
Published by darold almost 5 years ago
2019-10-29 - Version 3.1 released
This release is a maintenance release that adds support to PostgreSQL
v12.0. It also add a new report:
Table | Extended Statistic
public.cities | CREATE STATISTICS sch1.t1_stat (dependencies)
ON city,country FROM public.cities;
It also fixes some issues reported by users since last month:
Published by darold about 5 years ago
2019-09-18 - Version 3.0 released
This release is a major release that adds lot of improvement and
new reports. It also improve performances and fully support a CGI
mode with dynamic content for realtime reports.
New or enhanced reports/features:
New command line options:
New configuration directives :
It also fixes some issues reported by users since last ten months.
Published by darold almost 6 years ago
2018-12-13 - Version 2.9 released
This release is a minor release that fix a major issue in the
interval detection. This This led to a bad scale in bg_writter,
replication and most database statistics reports. Every one must
upgrade immediately.
It also fixes some issues reported by users since last release:
- Improve devices list retrieving and fix several menu issues.
- Fix checkpoint counter legend.
- Move query in pg_stat_statements at the right of the table for
better readability.
- Fix report of disk use, mounted devices, running processes,
crontab and installed package to avoid duplicate entries.
- Empty SysInfo arrays before reading sysinfo file, this prevent
duplicate data.
- Improve parsing of sar file.
- Fix interval that was set in milliseconds instead of second.
This conduct to wrong scale (ex: Kb instead of Gb) in bg_writter,
replication and database statistics reports. Thanks to Guillaume
Lelarge for the report.
Published by darold almost 6 years ago
2018-11-28 - Version 2.8 released
This release is a minor release that fix some issues reported by
users over past months, it also adds some new reports:
- Add disk space utilization report over time using df command
for sysstat version older than 11.1.4.
- Add report of percentage of disk space and inode used on each
file system. The information is available in sar output since
11.1.4 version.
- Add PageTables information in SysInfo memory report.
- Show database information about table/indexes in capture mode.
- Add report of number of tasks currently blocked, waiting for I/O
to complete "Run queue" report.
- Add new report about system dirty memory that need to be written
to disk as well as amount of active/inactive memory.
- Show highest dirty memory to write and highest number of processes
blocked in overall stats page.
- Add report of crontab entries for the user running pgcluu.
- Add report of installed PostgreSQL packages in main page.
New pgcluu_collectd command line option:
- Add command line option --package-list to be able to set a custom
command to list PostgreSQL packages. Default is to auto-detect
package type between rpm and dpkg, using command 'rpm -qa' or
'dpkg -l'. If you have an other system you can use this option to
set a custom command. A filter on keyword 'postgres' is appended
to the command: ' | grep postgres'.
Here are the complete list of bug fixes in this release:
- Change the way sysstat version is checked and save the version
into to sysinfo.txt file for use in pgcluu script.
- Remove extra new line at end of data file generated by the
patch on search_path securing.
- Add collect of disk space and inode used over time stored in file
fs_stat_use.csv. Need more work to avoid duplicate data with new
versions of sar that already report fs space use.
- Replace all call to Perl ternary operator with usual if-then-else
statement. In some condition this operator do not works and makes
pgcluu return negative values in overall stats. This was happening
only when pgcluu encounter a stat reset in the data files. Thanks
to Guillaume Lelarge for the report.
- Remove /dev/loop from devices statistics report.
- Add PageTables information in SysInfo memory report. Thanks to
Adrien Neyrat for the patch.
- Remove some unwanted metrics displayed in capture mode.
- Show database information about table/indexes in capture mode.
- Exclude /dev/loop from result of "df" command and squashfs from
"mount -l" command.
- Prevent pgcluu_collectd to stop before that all metrics collected
from the current loop have been written to disk. When it receives
the terminate signal (-k option) it previously stops immediately,
the problem was that this can conduct to empty or incomplete data
file. Now it waits until the loop is terminated before exiting.
- Fix compression of hourly directory when -R | --rotate-hourly and
-z | --compress options are enabled. The compression was called
each loop and not at each hour rotation.
- Add warning about mandatory use of -d option when pg_buffercache
reports are enabled (-B) but pg_buffercache extention is not found
in the connection database.
- Fix missing buffercache statistics since secure path has been
included in commit b61bf44. This required that pg_buffercache
table call was prefixed by the public schema. Thanks to
lobojohnson for the report.
- Fix unwanted print of tablespace path in cluster reports.
- Remove not useful information in buffercache graph legends.
- CSS: Increase size of the sysinfo panel.
- Adapt automatically pgCluu to changes in sar sysstat tool since
version 11.5.7. As per sysstat commit 8d635e0: Replace "rd_sec/s"
and "wr_sec/s" fields with "rkB/s" and "wkB/s". These fields are
now expressed in kilobytes instead of sectors. Replace "avgrq-sz"
field with "areq-sz". This field is now expressed in kilobytes
instead of sectors. Rename "avgqu-sz" field to "aqu-sz" to make
it consistent with iostat's output.
- Add report of number of tasks currently blocked, waiting for I/O
to complete to "Run queue" report. This information is available
with sysstat >= 9.1.7
- Show also highest dirty memory to write and highest number of
processes blocked in main page.
- Add new report about system dirty memory that need to be written
to disk as well as amount of active/inactive memory. These metrics
are available in recent sar/sysstat versions.
- Add report of crontab entries.
- Change icon for packages report.
- Add report of PostgreSQL installed package in main page.
- Add storage information about PostgreSQL packages installed on
the system.
- Add command line option --package-list to be able to set a custom
command to list PostgreSQL packages. Default is to auto-detect
package type between rpm and dpkg, using command 'rpm -qa' or
'dpkg -l'. If you have an other system you can use this option to
set a custom command. A filter on keyword 'postgres' is appended
to the command: ' | grep postgres'.
Published by darold over 6 years ago
2018-07-09 - Version 2.7 released
This release is a minor release that fix some issues reported by
users over past year but it also adds some new interesting reports:
- Add reports of pgbouncer wait for server statistics.
- Make pgCluu more compatible with PG10 and pgBouncer 1.8
- Add kernel vm.overcommit_kbytes configuration to report.
Here are the complete list of bug fixes in this release:
- Fix Prepared Transaction menu not disabled with no stat data.
- Fix typos and use proper measure in the graph. Thanks to Fabio Porta for the patch.
- Add vm.overcommit_kbytes to kernel report. Thanks to Adrien Neyrat for the patch.
- Since version 1.8 of pgbouncer, the command "show stats" changed output, make pgcluu
compatible. Thanks to Fabio Pardi for the report and the mapping of changes.
- Fix issue in partitionning lookup.
- Secure search_path before executing SQL queries.
- Fix several use of unintialized variable case.
- Add missing function IsLeapYear. Thanks to Emmanuel Boucle for the patch.
- Fix report for network and disk devices in incremental mode.
- Fix unwanted Network menu when no network device are found.
- Fetch devices informations. Thanks to Adrien Neyrat for the patch.
- Dockerise pgcluu, the container helps run both the collection and the report generator.
Thanks to Roy Golan for the patch.
- Disable System submenu when the corresponding report is not available.
- Append %idle to cpu report.
- Fix list of resources files. Thanks to Bosstek Consulting for the report.
- Change fetch_version() to be closer with the new versioning politic in 10.0 and above.
Thanks to Julien Rouhaud for the patch.
- Makes pgcluu more compatible with PostgreSQL 10.0.
Published by darold over 7 years ago
2017-07-09 - Version 2.6 released
This release is a minor release that fix some issues reported by
users over past year but it also adds some new interesting reports:
and useful features:
Here are the complete list of bug fixes in this release:
- Finalize systemd unit files. pgcluu_collectd.service is used to
start pgcluu_collectd in daemon mode. Other files, pgcluu.service
and pgcluu.timer to execute pgcluu periodically to generate reports.
- Add import of missing jqplot.canvasAxisLabelRenderer.min.js file used
to render axis labels in graphs.
- Set logo and icon on a single line in an url attribute as chrome
complain that it will be obsolete in M60.
- Add partitioning report and partition information into cache.
- Add hourly index when --rotate-hourly is enabled.
- Add support of partition information to cache mode.
- Fix unwanted exit in cache mode
- Cosmetic change in tooltip and download button. Main menu font size have also been improved.
- Update generated html to use Bootstrap 3 glyphicons.
- Add javascript and CSS sources, licences and download information for
packaging. Add a tool to minified and embedded the script and css into
pgcluu Perl script and copy the minified files into cgi-bin/rsc/
- Add minified resources file for CGI into cgi-bin/rsc
- Fix redundant index query. Thanks to Julien Rouhaud for the patch.
- Detect partitions and summarize information in a dedicated report.
Thanks to Julien Rouhaud for the patch.
- Refactor pidfile unlink handling. Thanks to Julien Rouhaud for
the patch.
- Limit call to pg_relation_size() when we are in capture mode.
Thanks to Guillaume Lelarge for the report.
- Add information about recheck of redundant index with primary key
and index on a column referencing a foreign key.
- Fix exclusion of UNIQUE index in redudant indexes report.
- Fix incremental global index on resize.
- Update year in copyright.
- Add index on main directory with incremental report to link to
the different days. Thanks to Heath Yob for the feature request.
- Add verification that pg_stat_statement is loaded from
shared_preload_libraries.
- Fix incomplete per-database-statistics. Thanks to Markus Braunig
for the report.
- Add kernel scheduler configuration for sched_autogroup_enabled
and sched_migration_cost_ns to sysinfo report. Thanks to Adrien
Nayrat for the patch.
- Fix broken main menu when no disk devices was present in sar
report.
- Allow to set label for y2axis in create_linegraph() parameters
- Change parameters and return of the get_diff() method to support
incremental mode.
- Create function get_diff() and shows_diff() to report
configuration file change.
- Append new color for line graph.
- Set missing database list with sysinfo and about menu.
- Fix Not a HASH reference in function reporting information about
indexes.
- Rename diff storage variables and stores them in binary file.
Initialyze storage variables for statistics that must be read
from file each time.
- Fix detection of working directories with dates over two months.
- Fix runqueue size report. Thanks to Thomas Reiss for the report.
- Create function to reuse the look for sysinfo file.
- pgcluu CGI now append new csv statistic to cached binary files.
- The menu is now built at end to avoid reading csv files first to
look for database, disk device and network interface.
- The CGI home page is now just built from the binary files to
speed up the first screen. As we do not append last data collected
from the csv files we indicate at which time the cache was last
built. This is to give a quick snapshot of the dashboard. When
looking at the statistics binary and csv files are both read.
If the cache have not been run yet, pgcluu will read dashboard
statistics from data fileis which takes longer to display.
- Allow incremental cache. When running in cache mode pgcluu will
append new csv statistics to old cached binary files. Then when
building reports, pgcluu will first load statistics from binary
files and complete statistics with data from csv file collected
after the last cache mode run. This mean that if cache mode is
run each minute on an incremental statistics collect (see option
-r or -R of pgcluu_collectd), pgcluu is able to create report very
quickly at any time. This is especially useful with CGI mode where
a full day statistics report can be displayed in few seconds.
- Fix diff of configuration files, old and new files was inversed.
Thanks to Adrien Nayrat for the report.
- Fix display of message no dataset on empty graphs.
- Update description of temporary files report and statistics
on checkpoints.
- Rewrite use of interval between collected data to support change
during collecting. Interval is now always calculated from the
difference between the current line end the previous line. This
mean that the first line is never present in reports with per
second statistics.
- Fix formatting of bytes in pretty number.
- Improve storage of database list with huge number of database.
- Fix timezone on all reports and remove graph per tablespace and
replace them with a list of tablespace and their location.
- Fix timezone on start and end timestamp of collect. Remove graph
per tablespace and replace them with a list of tablespace and
their location.
- Clear Start/End input box at startup in CGI.
- Comment and include REVERT_DATE toogle about sar date format
in CGI configuration file.
- Fix timezone and time selector in javascript menu.
- Fix use of timezone and date detection in sa file.
- Fix date parsing from sa text file.
- Fix some warning about use of undefined variables.
- Add link to last known statistics in CGI front page when no
data are found.
- Fix a call to timegm_nocheck()
- Split set_overall_stat_from_binary() in two function, one for
database orverall statistics and the other for system statistics.
- Fix parsing of sa file.
- Do not load statistic from the last day when hour:min is 00:00.
- Fix some bugs when there is just sar cached binary files.
- Fix two digit in date parts on start/end date.
- Fix cache mode when there is just sar file in entry.
- Add INCLUDE_DEV configuration directive to be able to filter
disk device. Allow comma as list separator in configuration file.
- Add information about how to parse sysstat sa binary and text file.
- Fix remove of unlogged tables following pg version.
- Fix colspan on unlogged table report.
- Add report of transaction throughput per second.
- Use psql -X option so .psqlrc doesn't get in the way. If .psqlrc
contains for example \timing, pgcluu_collectd will get confused.
Thanks to Christoph Berg for the patch.
- Separate svctm/await dataset on two different axes.
- Remove unwanted bgwriter stats prior pg 9.1
- Add data checksum status to CGI report.
- Fix autodetection of timezone and set default timezone for sar
statistics to the result of
perl -MPOSIX -e "print substr(strftime(\"%z\", localtime()), 0, 3);".
- Replace javascript call to dateToDisplay.toGMTString() with
dateToDisplay.toString(). Please note that this could not be
backward compatible with your previous timezone settings. See
pgcluu.js to revert the function call.
- Fix an issue with --from-sa-file where date was still incremented.
Thanks to Flavie Perette for the report.
- Replace sadf -D option by -d to obtain a compatible output.
- Separate svctm/await dataset on two different axes to be able to
see both dataset.
- Fix kernel parameter hugepage report when not available.
- Fix all call of psql that use both -f - and -c that doe not have
the same behavior in pg 9.6. Previous version does not care of
the -f stdin input when a command was provided with -c. This not
compatible anymore. Thanks to Vincent Laborie for the report.
- Fix collect of lock in pg_stat_activity with PostgreSQL 9.6.
Thanks to Vincent Laborie for the report.
Published by darold over 8 years ago
2016-04-27 - Version 2.5 released
This release is a major release that fix some issues reported by users over past year and a full replacement of the flotr2 javascript chart library with jqplot.
There's also some new interesting reports:
and useful features:
Note that a CGI script have been added to be able to perform temporal lookup in incremental pgCluu statistics, with predefined year, month, day and hour views. This is a work in progress, it will be available in next major release.
Upgrade: you can safely override previous installation, backward compatibility with 2.4 version is preserved.
See changelog for the complete list of changes in this release.