Bot releases are hidden (Show)
2023-12-25 - v12.4
This is a maintenance release of pgBadger that fixes issues reported by
users since last release.
Published by darold 11 months ago
2023-11-27 - v12.3
This is a maintenance release of pgBadger that fixes issues reported by
users since last release. It also adds some new features:
--include-pid
to only report events related to a session--include-session
to only report events related to the--dump-raw-csv
to only parse the log and dump the informationHere is the complete list of changes and acknowledgments:
Published by darold about 1 year ago
2023-08-20 - v12.2
This is a maintenance release of pgBadger that fixes issues reported by
users since last release. It also adds two new features:
Here is the complete list of changes and acknowledgments:
<...>
. Thanks to Fabio Geiss for the report.system usage: CPU: ...
line parsing. Thanks to Francisco Reinolds for the patch.%p
and %t
as mandatory placeholders log information. Thanks to Christophe Courtois for the patch.POSIX::strftime("%s", ....)
by a call to localtime for Windows port. Thanks to Bertrand Bourgier for the patch.Published by darold over 1 year ago
2023-03-20 - v12.1
This is a maintenance release of pgBadger that fixes issues reported by users since past six months.
Here is the complete list of changes and acknowledgments:
Published by darold about 2 years ago
2022-09-13 - v12.0
This major release of pgBadger fixes some issues reported by users since
past five months. As usual there is also new features and improvements:
Here is the complete list of changes and acknowledgments:
Published by darold over 2 years ago
2022-04-08 - v11.8
This release of pgBadger fix some issues reported by users since past three months and especially two fixes on new log entries detection in incremental mode.
Here is the complete list of changes and acknowledgments:
- Full review and simplification of the log file change detection.
- Reports messages "could not (receive|send) data (from|to) client" in the
Events reports. Thanks to Adrien Nayrat for the report.
- Fix parsing issue when the name of a prepared query contain the ':'
character. Thanks to aleszeleny for the report.
- Fix detection of new log entries with timestamp when millisecond (%m) or
epoch (%n). Thanks to aleszeleny for the report.
- Fix detection of new log entries in local file when multiprocess was not
used. Thanks to aleszeleny for the report.
- Fix detection of new log entries in remote files through ssh. Thanks to
Luca Ferrari for the report
- Fix garbage in username of "Connections per user" report. Thanks to
caseyandgina for the report.
- Fix ssh command when using URI, the ssh options was missing. Thanks to Luca
Ferrari for the report.
- Handle queryid %Q placeholder. Thanks to Adrien Nayrat for the patch.
- Fix typo in error sentence. Thanks to Luca Ferrari for the patch
- Report message: "server process was terminated by signal" in the Events
report. Thanks to Avi Vallarapu for the report.
- doc: fix filename for incremental every week command. Thanks to Theophile
Helleboid for the patch.
- t/04_advanced.t: Fix syslog test. Thanks to Christoph Berg for the patch.
Published by darold over 2 years ago
2022-01-23 - v11.7
This release of pgBadger fix some issues reported by users since past five months as well as some improvements:
--no-progressbar
option to not display it but keep the other outputs.--day-report
that can be used to rebuild an HTML report over the specified day. Like option --month-report
but only for a day. It requires the incremental output directories and the presence of all necessary binary data files. The value is date in format: YYYY-MM-DDHere is the complete list of changes and acknowledgments:
Published by darold about 3 years ago
2021-09-04 - v11.6
This release of pgBadger fix some issues reported by users since past seven months as well as some improvements:
--explode
is used. Thanks to MigOps Inc for the patch and Thomas Leclaire for the feature request.--iso-week-number
in incremental mode, calendar's weeks start on a Monday and respect the ISO 8601 week number, range 01 to 53, where week 1 is the first week that has at least 4 days in the new year. Thanks to Alex Muntada for the feature request.Important note:
--iso-week-number
will be the default in next major release and that --start-monday
option will be removed as the week will always start a Monday. The possibility to have week reports start a Sunday will be removed to simplify the code.Here is the complete list of changes and acknowledgments:
- Fix duplicate of warning message:
"database ... must be vacuumed within ... transactions".
Thank to Christophe Courtois for the report.
- Fix use of uninitialized variable. Thanks to phiresky for the report.
- Improve query id detection, it can be negative, as well as read it
from csvlog.
- Fix case where last file in incremental mode is always parsed even if
it was already done. Thanks to Thomas Leclaire for the report.
- Update syslog format regex to handle where session line indicator
only contains one int vs two ints separated by dash. Thanks to
Timothy Alexander for the patch.
- Fix --exclude-db option to create anyway the related report with json
log. Thanks to MigOps Inc for the patch and Thomas Leclaire for the
report.
- Add regression test about Storable buggy version.
- Fix use of uninitialized value in substitution iterator in incremental
mode during the week report generation. Thanks to Thomas Leclaire,
Michael Vitale, Sumeet Shukla and Stefan Corneliu Petrea for the report.
- Add 'g' option to replace all bind parameters. Thanks to Nicolas Lutic
and Sebastien Lardiere for the patch.
- Documentation improvements. Thanks to Stefan Petrea for the patch.
- Fixes change log time zone calculation. Thanks to Stefan Petrea for
the patch.
- Fix log filter by begin/end time.
- Fix wrong association of orphan lines for multi-line queries with a
filter on database. Thanks to Abhishek Mehta for the report.
- Fix reports in incremental mode when --dbname parameter is partially
ignored with "explode" option (-E). Thanks to lrevest for the report.
- Update javascript resources.
- Fix display of menu before switching to hamburger mode when screen is
reduced. Thanks to Guillaume Lelarge for the report.
- Fix bind parameters values over multiple lines in the log that were
not well supported.
- Apply same fix for previous patch than in pgFormatter.
- Fix an other use of uninitialized value in substitution iterator from
pgFormatter code. Thanks to Christophe Courtois for the report.
- Fix query normalization. Thanks to Jeffrey Beale for the patch.
- Be sure that all statements end with a semicolon when --dump-all-queries
is used. Thanks to Christian for the report.
- Fix typo and init of EOL type with multiple log files.
- Add auto detection of EOL type to fix LAST_PARSED offset when OEL is on
2 bytes (Windows case). Thanks to Bertrand Bourgier for the patch.
- Fix get_day_of_week() port on Windows where strftime %u is not supported.
Thanks to Bertrand Bourgier for the patch.
- Fix Windows port that call pl2bat.bat perl utility to create a corrupted
pgbadger.bat du to the way __DATA__ was read in pgbadger. Thanks to
Bertrand Bourgier for the patch.
- Fix begin/end time filter and add regression test for timestamp filters.
Thanks to Alexis Lahouze and plmayekar for the report.
- Fix use of uninitialized value in pattern match introduced by pgFormatter
update. Thanks to arlt for the report.
Published by darold over 3 years ago
2021-02-18 - v11.5
This release of pgBadger fix some issues reported by users since
past three months as well as some improvements:
Here is the complete list of changes and acknowledgments:
Published by darold almost 4 years ago
2020-11-24 - v11.4
This release of pgBadger fix some issues reported by users since
past four months. Improve support for PostgreSQL 13 log information
and adds some new features:
Here is the complete list of changes and acknowledgments:
- Fix typo in an error message. Thanks to Vidar Tyldum for the patch.
- Fix Windows port with error: "can not load incompatible binary data". Thanks to Eric Brawner for the report.
- Fix typo on option --html-outdir in pgbadger usage and documentation. Thanks to Vidar Tyldum for the patch.
- Fix autodetection of jsonlog/cloudsql format. Thanks to Jon Young for the report.
- Fix CSV log parsing with PG v13. Thanks to Kanwei Li for the report and Kaarel Moppel for the patch.
- Fix sort of queries generating the most temporary files report. Thanks to Sebastien Lardiere for the report.
- Add pgbadger version trace in debug mode.
Published by darold about 4 years ago
2020-07-26 - v11.3
This release of pgBadger fix several issues reported by users since
past months. It also adds some new features and new command line
options:
* Add autodetection of UTC timestamp to avoid applying timezone
for graphs.
* Add support to GCP CloudSQL json log format.
* Add new option --dump-all-queries to use pgBadger to dump all
queries to a text file, no report is generated just the full list
of statements found in the PostgreSQL log. Bind parameters are
inserted into the queries at their respective position.
* Add new option -Q | --query-numbering used to add numbering of
queries to the output when using options --dump-all-queries or
--normalized-only.
* Add new command line option --tempdir to set the directory where
temporary files will be written. Can be useful on system that do
not allow writing to /tmp.
* Add command line option --ssh-port used to set the ssh port if not
default to 22. The URI notation also adds support to ssh port
specification by using the form:
ssh://192.168.1.100:2222//var/log/postgresql-11.log
Here is the complete list of changes and acknowledgments:
- Fix incremental reports for jsonlog/cloudsql log format. Thanks
to Ryan DeShone for the report
- Add autodetection of UTC timestamp to avoid applying autodetected
timezone for graphs. With UTC time the javascript will apply the
local timezone. Thanks to Brett Stauner for the report.
- Fix incremental parsing of journalctl logs doesn't work from the
second run. Thanks to Paweł Koziol for the patch.
- Fix path to resources file when -X and -E are used. Thanks to Ryan
DeShone for the report.
- Fix General Activity report about read/write queries. Thanks to
alexandre-sk5 for the report.
- Add debug message when parallel mode is not use.
- Fix elsif logic in file size detection and extra space introduced
in the journalctl command when the --since option is added. Thanks
to Pawel Koziol for the patch.
- Fix "not a valid file descriptor" error. Thanks to Pawel Koziol
for the report.
- Fix incremental mode with RDS files. Thanks to Ildefonso Camargo,
nodje and John Walsh for the report.
- Add new option -Q | --query-numbering used to add numbering of
queries to the output when using options --dump-all-queries or
--normalized-only. This can be useful to extract multiline queries
in the output file from an external script. Thanks to Shantanu Oak
for the feature request.
- Fix parsing of cloudsql json logs when log_min_duration_statement
is enabled. Thanks to alexandre-sk5 for the report.
- Fix wrong hash key for users in RDS log. Thanks to vosmax for the
report.
- Fix error related to modification of non-creatable array value.
Thanks to John Walsh and Mark Fletcher for the report.
- Add support to GCP CloudSQL json log format, log format (-f) is
jsonlog. Thanks to Thomas Poindessous for the feature request.
- Add new option --dump-all-queries to use pgBadger to dump all
queries to a text file, no report is generated just the full list
of statements found in the PostgreSQL log. Bind parameters are
inserted into the queries at their respective position. There is
not sort on unique queries, all queries are logged. Thanks to
Shantanu Oak for the feature request.
- Add documentation for --dump-all-queries option.
- Fix vacuum report for new PG version. Thanks to Alexey Timanovsky
for the report.
- Add new command line option --no-process-info to disable change of
process title to help identify pgbadger process, some system do
not allow it. Thanks to Akshay2378 for the report.
- Add new command line option --tempdir to set the directory where
temporary files will be written. Default:
File::Spec->tmpdir() || '/tmp'
Can be useful on system that do not allow writing to /tmp. Thanks
to Akshay2378 for the report.
- Fix unsupported compressed filenames with spaces and/or brackets.
Thanks to Alexey Timanovsky for the report.
- Add command line option --ssh-port used to set the ssh port if not
default to 22. The URI notation also adds support to ssh port
specification by using the form:
ssh://192.168.1.100:2222//var/log/postgresql-11.log
Thanks to Augusto Murri for the feature request.
Published by darold over 4 years ago
2020-03-11 - v11.2
This release of pgBadger fix several issues reported by users since
past six months. It also adds some new features:
* Add support and autodetection of AWS redshift log format.
* Add support to pgbouncer 1.11 new log format.
* Handle zstd and lz4 compression format
* Allow to fully separate statistics build and HTML report build in
incremental mode without having to read a log file. For example
it is possible to run pgbadger each hours as follow:
pgbadger -I -O "/out-dir/data" --noreport /var/log/postgresql*.log
It just creates the data binary files in "/out-dir/data" then
for example you can make reports each night for the next day in
a separate directory `/out-dir/reports`:
pgbadger -I -l "/out-dir/data/LAST_PARSED" -H "/out-dir/reports" /out-dir/data/2020/02/19/*.bin
This require to set the path to the last parsed information, the
path where HTML reports will be written and the binary data file
of the day.
There is also new command line options:
* Add new command line option --explain-url used to override the url
of the graphical explain tool. Default URL is:
http://explain.depesz.com/?is_public=0&is_anon=0&plan=
If you want to use a local install of PgExplain or an other tool.
pgBadger will add the plan in text format escaped at the end of
the URL.
* Add new option --no-week to instruct pgbadger to not build weekly
reports in incremental mode. Useful if it takes too much time and
resources.
* Add new command line option --command to be able to set a command
that pgBadger will execute to retrieve log entries on stdin.
pgBadger will open a pipe to the command and parse log entries
generated by the command. For example:
pgbadger -f stderr --command 'cat /var/log/postgresql.log'
which is the same as executing pgbadger with the log file directly
as argument. The interest of this option is obvious if you have to
modify the log file on the fly or that log entries are extracted
from a program or generated from a database. For example:
pgbadger -f csv --command 'psql dbname -c "COPY jrn_log TO STDOUT (FORMAT CSV)"'
* Add new command line option --noexplain to prevent pgBadger to
parse and report explain plan written to log by auto_explain
extension. This is useful if you have a PostgreSQL version < 9.0
where pgBadger generate broken reports when there is explain plan
in log.
Backward compatibility:
- By default pgBadger will truncate queries up to 100000 characters.
This arbitrary value and can be adjusted using option --maxlength.
Previous behavior was to not truncate queries but this could
lead in excessive resources usage. Limiting default size is safer
and the size limit might allow no truncate in most cases. However
queries will not be beautified if they exceed 25000 characters.
Here is the complete list of changes and acknowledgments:
- Fix non working --exclude-client option. Thanks to John Walsh
for the report.
- Add regression test for RDS log parsing and --exclude-client.
- Fix progress bar for pgbouncer log file. The "queries" label is
changed in "stats" for pgbouncer log files.
- Add command line option --explain-url used to override the url
of the graphical explain tool. Thanks to Christophe Courtois for
the feature request.
- Add support to pgbouncer 1.11 new log format. Thanks to Dan
Aksenov for the report.
- Handle zstd and lz4 compression format. Thanks to Adrien Nayrat
for the patch.
- Add support and autodetection of AWS redshift log format. Thanks
to Bhuvanesh for the reature request.
- Update documentation about redshift log format.
- Add new option --no-week to instruct pgbadger to not build weekly
reports in incremental mode. Thanks to cleverKermit17 for the
feature request.
- Fix a pattern match on file path that breaks pgBadger on Windows.
- Fix #554 about cyrillic and other encoded statement parameters
that was not reported properly in the HTML report even with custom
charset. The regression was introduced with a fix to the well
known Perl error message "Wide character in print". The patch have
been reverted and a new command line option: --wide-char is
available to recover this behavior. Add this option to your
pgbadger command if you have message "Wide character in print".
Add a regression test with Cyrillic and french encoding. Thanks
to 4815162342lost and yethee for the report.
- Update documentation to inform that lc_messages = 'en_US.UTF-8'
is valid too. Thanks to nodje for the report.
- Update documentation about --maxlength which default truncate size
is 100000 and no more default to no truncate. Thanks to nodje for
the report.
- Fix retention calculation at year overlap. Thanks to Fabio Pereira
for the patch.
- Fix parsing of rds log file format. Thanks to Kadaffy Talavera for
the report.
- Prevent generating empty index file in incremental mode when there
is no new log entries. Thanks to Kadaffy Talavera for the report.
- Fix non up to date documentation. Thanks to Eric Hanson for the
patch.
- Fixes the command line parameter from -no-explain to -noexplain.
Thanks to Indrek Toom for the patch.
- Fall back to default file size when totalsize can not be found.
Thanks to Adrien Nayrat for the patch.
- Fix some dates in examples. Thanks to Greg Clough for the patch.
- Use compressed file extension regexp in remaining test and extract
.bin extension in a separate condition.
- Handle zstd and lz4 compression format. Thanks to Adrien Nayrat
for the patch.
- Fix remaining call of SIGUSR2 on Windows. Thanks to inrap for the
report.
- Fix progress bar with log file of indetermined size.
- Add new command line option --command to be able to set a command
that pgBadger will execute to retrieve log entries on stdin.
Thanks to Justin Pryzby for the feature request.
- Add new command line option --noexplain to prevent pgBadger to
parse and report explain plan written to log by auto_explain
extension. This is useful if you have a PostgreSQL version < 9.0
where pgBadger generate broken reports when there is explain plan
in log. Thanks to Massimo Sala for the feature request.
- Fix RDS log parsing when the prefix is set at command line. Thanks
to Bing Zhao for the report.
- Fix incremental mode with rds log format. Thanks to Bing Zhao for
the report.
- Fix possible rds log parsing. Thanks to James van Lommel and Simon
Dobner for the report.
- Fix statement classification and add regression test. Thanks to
alexanderlaw for the report.
- Fix anonymization of single characters in IN clause. Thanks to
Massimo Sala for the report.
- Fix RDS log parsing for rows without client/user/db information.
Thanks to Konrad for the report.
Published by darold about 5 years ago
2019-09-16 - v11.1
This release of pgBadger fix several issues reported by users since
three months. It also adds some new features and reports:
Total query duration: 6m16s
Prepare/parse total duration: 45s564ms
Bind total duration: 4m46s
Execute total duration: 44s71m
-f rds
if pgbadger is not able to auto-detect the log format.--month-report
to be able to build monthly incremental reports.There's also some bugs fixes and features enhancements.
- Add auto-generated Markdown documentation in README.md using tool
pod2markdown. If the command is not present the file will just not
be generated. Thanks to Derek Yang for the patch.
- Translate action WITH into CTE, regression introduced in last release.
- Fix support of Windows Operating System
- Add support to RDS and CloudWatch log format, use -f rds if pgbadger is
not able to auto-detect this log format. Thanks to peruuparkar for the
feature request.
- Fix option -f | --format that was not applied on all files get from the
parameter list where log format auto-detection was failing, the format was
taken from the fist file parsed. Thanks to Levente Birta for the report.
- Update source documentation file to replace reference to pgBadger v7.x
with v11. Thanks to Will Buckner for the patch.
- Limit height display size of top queries to avoid taking the whole page
with huge queries. Thanks to ilias ilisepe1 for the patch.
- Fix overflow of queries and detail in Slowest individual queries.
- Fix SSH URIs for files, directories and wildcards. Thanks to tbussmann for
the patch.
- Fix URI samples in documentation. Thanks to tbussmann for the patch.
- Hide message of use of default out file when --rebuild is used.
- Add extra newline to usage() output to not bread POD documentation at
make time.
- Reapply --exclude-client option description in documentation. Thanks to
Christoph Berg for the report.
Published by darold over 5 years ago
2019-06-25 - v11.0
This release of pgBadger adds some major new features and fixes some
issues reported by users since the last four months. New features:
Regroup cursor related query (DECLARE,CLOSE,FETCH,MOVE) into new
query type CURSOR.
Add top bind queries that generate the more temporary files.
Require log_connection and log_disconnection be activated.
Add --exclude-client command line option to be able to exclude log
entries for the specified client ip. Can be used multiple time.
Allow to use time only in --begin and --end filters.
Add -H, --html-dir option to be able to set a different path where
HTML report must be written in incremental mode. Binary files stay
on directory defined with -O, --outdir option.
Add -E | --explode option to explode the main report into one
report per database. Global information not related to a database
are added to the postgres database report.
Add per database report to incremental mode. In this mode there
will be a sub directory per database with dedicated incremental
reports.
Add support to Heroku's PostgreSQL logplex format. Log can be
parsed using:
heroku logs -p postgres | pgbadger -f logplex -o heroku.html -
When a query is > 10Kb we first limit size of all constant string
parameters to 30 characters and then the query is truncated to 10Kb.
This prevent pgbadger to waste time/hang with very long queries
when inserting bytea for example. The 10Kb limit can be controlled
with the --maxlength command line parameter.
The query is normalized or truncated to maxlength value only after
this first attempt to limit size.
This new release breaks backward compatibility with old binary or JSON
files. This also mean that incremental mode will not be able to read
old binary file. If you want to update pgBadger and keep you old reports
take care to upgrade at start of a new week otherwise weekly report will
be broken. pgBadger will print a warning and just skip the old binary
file.
There's also some bugs fixes and features enhancements.
- Add a warning about version and skip loading incompatible binary file.
- Update code formatter to pgFormatter 4.0.
- Fix pgbadger hang on Windows OS. Thanks to JMLessard for the report.
- Update tools/pgbadger_tools script to be compatible with new binary
file format in pgBadger v11.
- Add top bind queries that generate the more temporary files. This
collect is possible only if log_connection and log_disconnection
are activated in postgresql.conf. Thanks to Ildefonso Camargo for
the feature request.
- Fix auto detection of timezone. Thanks to massimosala for the fix.
- Remove some remaining graph when --nograph is used
- Force use of .txt extension when --normalized-only is used.
- Fix report of auto vacuum/analyze in logplex format. Thanks to
Konrad zichul for the report.
- Fix use of progress bar on Windows operating system. Thanks to
JMLessard for the report.
- Use a `$prefix_vars{'t_time'} to store the log time. Thanks to Luca
Ferrari for the patch.
- Update usage and documentation to remove perl command from pgbadger
invocations. Thanks to Luca Ferrari for the patch.
- Use begin and end with times without date. Thanks to Luca Ferrari
for the patch.
- Added some very minor spelling and grammar fixes to the readme file.
Thanks to ofni yratilim for the patch.
- Fix remote paths using SSH. Thanks to Luca Ferrari for the patch.
- Update regression test to works with new structure introduced with
the per database report feature.
- Fix fractional seconds in all begin and end parameters. Thanks to
Luca Ferrari for the patch.
- Fix documentation URL. Thanks to Kara Mansel for the report.
- Fix parsing of auto_explain.
Add more information about -U option that can be used multiple time.
Thanks to Douglas J Hunley for the report.
- Lot of HTML / CSS report improvements. Thanks to Pierre Giraud for
the patches.
- Update resource file.
- Add regression test for logplex format.
- Add support to Heroku's PostgreSQL logplex format. You should be able
to parse these logs as follow:
heroku logs -p postgres | pgbadger -f logplex -o heroku.html -
or if you have already saved the output to a file:
pgbadger heroku.log
The logplex format is auto-dectected like any other supported format.
pgBadger understand the following default log_line_prefix:
database = %d connection_source = %r sql_error_code = %e
or simply:
sql_error_code = %e
Let me know if there's any other default log_line_prefix. The prefix
can always be set using the -p | --prefix pgbadger option:
pgbadger --p 'base = %d source = %r sql_state = %e' heroku.log
for example.
Thanks to Anthony Sosso for the feature request.
- Fix pgbadger help on URI use.
- Fix broken wildcard use in ssh URI introduced in previous patch.
Thanks to Tobias Bussmann for the report.
- Allow URI with space in path to log file. Thanks to Tobias Bussmann
for the report.
- Fix URI samples in documentation. Thanks to Tobias Bussmann for the
patch.
- Fix t/02_basics.t to don't fail if syslog test takes more than 10s.
Thanks to Christoph Berg for the patch.
Published by darold over 5 years ago
2019-02-14 - v10.3
This release of pgBadger is a maintenance release that fixes some
log format autodetection issues another pgBouncer log parsing issue
reported by users. There is also a new feature:
The -o | --outfile option can now be used multiple time to dump
output in several format in a single command. For example:
pgbadger -o out.html -o out.json /log/pgsql-11.log
will create two reports in html and json format saved in the
two corresponding files.
There's also some bugs fixes and features enhancements.
- Fix statistics reports when there a filter on database, user,
client or application is requested. Some queries was not
reported.
- Fix autodetection of pg>=10 defauilt log line prefix.
- Fix autodetection of log file with "non standard" log line prefix.
If --prefix specify %t, %m, %n and %p or %c, set format to stderr.
Thanks to Alex Danvy for the report.
- Remove extra space at end of line.
- Add minimal test to syslog parser.
- Fix a call to autodetect_format().
- Truncate statement when maxlength is used. Thanks to Thibaud
Madelaine for the patch.
- Add test for multiple output format.
- The -o | --outfile option can now be used multiple time to dump
output in several format in a single command. For example:
pgbadger -o out.txt -o out.html -o - -x json /log/pgsql-11.log
Here pgbadger will create two reports in text and html format
saved in the two corresponding file. It will also output a json
report on standard output. Thanks to Nikolay for the feature
request.
- Move detection of output format and setting of out filename into
a dedicated function set_output_extension().
- Fix another pgBouncer log parsing issue. Thanks to Douglas J.
Hunley for the report.
Published by darold almost 6 years ago
2018-12-27 - v10.2
This release of pgBadger is a maintenance release that fixes issues
reported by users during last three months. There is also some new
features:
* Add support to pgbouncer 1.8 Stats log format.
* Auto adjust javascript graph timezone.
* Add support to http or ftp remote PostgreSQL log file download.
The log file is parsed during the download using curl command
and never saved to disk. With ssh remote log parsing you can use
uri as command line argument to specify the PostgreSQL log file.
ssh://localhost/postgresql-10-main.log
http://localhost/postgresql-10-main.log.gz
ftp://localhost/postgresql-10-main.log
with http and ftp protocol you need to specify the log file format
at end of the uri:
http://localhost/postgresql-10-main.log:stderr
You can specify multiple uri for log files to be parsed. This is
useful when you have pgbouncer log file on a remote host and
PostgreSQL logs in the local host.
With ssh protocol you can use wild card too like with remote
mode, ex: ssh://localhost/postgresql-10-main.log*
Old syntax to parse remote log file using -r option is still
working but is obsolete and might be removed in future versions.
There is a new command line option:
* Add --exclude-db option to compute report about everything except
the specified database.
There's also some bugs fixes and features enhancements.
- Adjust end of progress bar with files with estimate size (bz2
compressed files and remote compressed files.
- Update year in copyright.
- Add information about URI notation to parse remote log files.
- Force progress to reach 100% at end of parsing of compressed
remote file.
- Extract information about PL/pgSQL function call in queries of
temporary file reports. The information is append to the details
display block.
- Fix progress bar with csv files.
- Fix reading binary file as input file instead of log file.
- Encode html output of queries into UTF8 to avoid message "Wide
character in print". Thanks to Colin 't Hart for the report.
- Add Checkpoints distance key/value for distance peak.
- Fix pgbouncer parsing and request throughput reports. Thanks
to Levente Birta for the report.
- Fix use of csvlog instead of csv for input format.
- Add support to pgbouncer 1.8 Stats log format. Thanks to Levente
Birta for the report.
- Add warning about parallel processing disabled with csvlog. Thanks
to cstdenis for the report.
- Add information in usage output about single process forcing with
csvlog format in -j and -J options. Thanks to cstdenis for the
report.
- Fix unknown line format error for multi line log while incremental
analysis over ssh. Thanks to Wooyoung Cho for the report.
- Add -k (--insecure) option to curl command to be able to download
logs from server using a self signed certificate.
- Auto adjust javascript graph timezone. Thanks to Massimino Sala
for the feature request.
- Add support to HTTP logfile download by pgBadger, for example:
/usr/bin/pgbadger http://www.mydom.com/postgresql-10.log
- Will parse the file during download using curl command.
- Fix documentation. Thanks to 0xflotus for the patch.
- Reapply fix on missing replacement of bind parameters after some
extra code cleaning. Thanks to Bernhard J. M. Grun for the report.
- Add --exclude-db option to compute report about everything except
the specified database. The option can be used multiple time.
Published by darold about 6 years ago
2018-09-12 - v10.1
This release of pgBadger is a maintenance release that fixes reports
in incremental mode and multiprocess with -j option. Log parsing from
standard input was also broken. If you are using v10.0 please upgrade
now.
Thanks a lot to Etienne Bersac from Dalibo who have initiate the
pgbadger test suite and added information about contributing in
the documentation. Unitary tests are still not complete but tests
of features fixed in this maintenance release have been added.
Published by darold about 6 years ago
2018-09-09 - v10.0
This release of pgBadger is a major release that adds some new
features and fix all issues reported by users since last release.
There is some new command line option:
The pgBadger project and copyrights has been transfered from Dalibo
to the author and official maintainer of the project. Please update
your links:
I want to thanks the great guys at Dalibo for all their investments
into pgBadger during these years and especially Damien Clochard and
Jean-paul argudo for their help to promote pgBadger.
Here is the complete list of changes and acknowledgement in this
release.
- Fix checkpoint distance and estimate not reported in incremental
mode. Thanks to aleszeleny for the report.
- Fix title of pgbouncer simultaneous session report. Thanks to
Jehan Guillaume De Rorthais for the report.
- Add support of pgbouncer syslog log file format. Thanks to djester
for the feature request.
- Fix error when a remote log is empty. Thanks to Parasit Hendersson
for the report.
- Fix test with binary format. Binary file must be generated as it
is dependent of the plateform. Thanks to Michal Nowak for the
report.
- Fix case where an empty explain plan is generated.
- Fix parsing of autodetected default format with prefix in command
line.
- Remove dependency of git command in Makefile.PL.
- Update documentation about options changes and remove of the
[%l-1] part of the mandatory prefix.
- Fix parsing of vacuum / analyze system usage for PostgreSQL 10.
Thanks to Achilleas Mantzios for the patch.
- Fix Temporary File Activity table.
- Remove dependency to git during install.
- Add --log-timezone +/-XX command line option to set the number
of hours from GMT of the timezone that must be used to adjust
date/time read from log file before beeing parsed. Using this
option make more difficult log search with a date/time because the
time will not be the same in the log. Note that you might still
need to adjust the graph timezone using -Z when the client has not
the same timezone. Thanks to xdexter for the feature request and
Julien Tachoire for the patch.
- Add support to auto_explain json output format. Thanks to dmius
for the report.
- Fix auto_explain parser and queries that was counted twice.
Thanks to zam6ak for the report.
- Fix checkpoint regex to match PostgreSQL 10 log messages. Thanks
to Edmund Horner for the patch.
- Update description of -f | --format option by adding information
about jsonlog format.
- Fix query normalisation to not duplicate with bind queries.
Normalisation of values are now tranformed into a single ? and no
more 0 for numbers, two single quote for string. Thanks to vadv
for the report.
- Fix log level count. Thanks to Jean-Christophe Arnu for the report
- Make pgbadger more compliant with B::Lint bare sub name.
- Made perlcritic happy.
- Add --prettify-json command line option to prettify JSON output.
Default output is all in single line.
- Fix Events distribution report.
- Fix bug with --prefix when log_line_prefix contain multiple %%.
Thanks to svb007 for the report.
- Add --log-timezone +/-XX command line option to set the number
of hours from GMT of the timezone that must be used to adjust
date/time read from log file before beeing parsed. Using this
option make more difficult log search with a date/time because the
time will not be the same in the log. Note that you might still
need to adjust the graph timezone using -Z when the client has not
the same timezone. Thanks to xdexter for the feature request.
- Remove INDEXES from the keyword list and add BUFFERS to this list.
- Fix normalization of query using cursors.
- Remove Dockerfile and documentation about docker run. pgBadger
comes as a single Perl script without any dependence and it can
be used on any plateform. It is a non sens to use docker to run
pgbadger, if you don't want to install anything, just copy the
file pgbadger where you want and execute it.
- Fix broken grid when no temp files activity. Thanks to Pierre
Giraud for the patch
- Add doc warning about log_in_duration_statement vs log_duration +
log_statement. Thanks to Julien Tachoire for the patch.
- Apply timezone offset to bar charts. Thanks to Julien Tachoire
for the patch.
- Delete current temp file info if we meet an error for the same PID
Thanks to Julien Tachoire for the patch.
- Consistently use app= in examples, and support appname=
Some of the usage examples used appname= in the prefix, but the
code didn't recognize that token. Use app= in all examples, and
add appname= to the prefix parser. Thanks to Christoph Berg for
the patch
- Fix wrong long name for option -J that should be --Jobs intead
of --job_per_file. Thanks to Chad Trabant for the report and
Etienne Bersac for the patch.
- Ignore blib files. Thanks to Etienne Bersac for the patch.
- Add consistency tests. Thanks to damien clochard for the patch.
- doc update : stderr is not a default for -f. Thanks to Christophe
Courtois for the patch.
- Always update pod and README. Thanks to Etienne Bersac for
the patch.
- Add some regression tests. Thanks to Etienne Bersac for the patch.
- Add editorconfig configuration. Thanks to Etienne Bersac for the
patch.
- Drop vi temp files from gitignore. Thanks to Etienne Bersac for
the patch.
- Add --include-time option to add the ability to choose times that
you want to see, instead of excluding all the times you do not
want to see. This is handy when wanting to view only one or two
days from a week's worth of logs (simplifies down from multiple
--exlucde-time options to one --include-time). Thanks to Wesley
Bowman for the patch.
- Check pod syntax. Thanks to Etienne Bersac for the patch.
- Add HACKING to document tests. Thanks to Etienne Bersac for the
patch.
- Drop obsolete --bar-graph option. Thanks to Etienne Bersac for
the patch.
- Drop misleading .perltidyrc. This file date from 2012 and
pgbadger code is far from compliant. perltidy unified diff is
10k lines. Let's drop this. Thanks to Etienne Bersac for the
patch.
- Fix use of uninitialized value in SQL formatting. Thanks to John
Krugger for the report and Jean-paul Argudo for the report.
Published by darold about 7 years ago
This release of pgBadger is a maintenance release that adds some new
features.
There's also some bugs fixes and features enhancements.
- Fix reports with histogram that was not showing data upper than
the last range.
- Fix parsing of journalctl without the the log line number pattern
([%l-n]). Thanks to Christian Schmitt for the report.
- Add report of checkpoint distance and estimate. Thanks to jjsantam
for the feature request.
- Append more information on what is done by script to update CSS
and javascript files, tools/updt_embedded_rsc.pl.
- Do not warn when all log files are empty and exit with code 0.
- Fix build_log_line_prefix_regex() that does not include %n as a
lookup in %regex_map. Thanks to ghosthound for the patch.
- Change error level of "FATAL: cannot use CSV" to WARNING. Thanks
to kong1man for the report.
- Fix use of uninitialized value warning. Thanks to Payal for the
report.
- Add permission denied to error normalization
- Update pgbadger to latest commit 5bdc018 of pgFormatter.
- Add support for AWS Redshift keywords. Thanks to cavanaug for the
feature request.
- Fix missing query in temporary file report when the query was
canceled. Thanks to Fabrizio de Royes Mello for the report.
- Normalize query with binded parameters, replaced with a ?.
- Sanity check to avoid end time before start time. Thanks to
Christophe Courtois for the patch.
- Fix a lot of mystyped words and do some grammatical fixes. Use
'pgBadger' where it refers to the program and not the binary file.
Also, use "official" expressions such as PgBouncer, GitHub, and
CSS. POD file was synced with README. Thanks to Euler Taveira for
the patch.
- Menu is broken when --disable-type top_cancelled_info test and
closing list must be inside disable_type test. While in it, ident
disable_lock test. Thanks to Euler Taveira for the patch.
- Fix use of uninitialized value. Thanks to johnkrugger for the
report.
- Remove test to read log file during log format auto-detection when
the file is hosted remotly. Thanks to clomdd for the report.
- Add autodetection of log format in remote mode to allow remote
parsing of pgbouncer log file together with PostgreSQL log file.
- Fix number of sessions wrongly increased after log line validation
Thanks to Achilleas Mantzios for the report.
- Minor reformatting of the pgBadger Description.
- Fix repeated info in documentation. Thanks to cscatolini for the patch.
Published by darold over 7 years ago
2017-01-24 - v9.1
This release of pgBadger is a maintenance release that adds some new
features.
There's also some bugs fixes and features enhancements.