libpqxx

Here's what's new in this release:

• Fix CMake documentation install. (#848)
• Bump gcc/clang/postgres minimum version requirements.
• Another fix to the readthedocs documentation build. (#845)
• Remove obsolete CMake workaround kludges. (#851, #866, #867, #869)
• Remove obscure deprecated stream_to constructor that never worked. (#853)
• Support reading a field as an SQL array using as_sql_array(). (#841)
• Make row iterator a proper random_access_iterator. (#846)
• Downgrade result iterator to "bidirectional"; was never really RA. (#846)

You may wonder why result iterators are no longer random access iterators. Turns out they never were! That's because if you have a result iterator i, referring to a row, then i[n] in libpqxx refers to field n in that row. The definition for random access iterators expects it to mean *(i+n) instead.

Documentation on ReadTheDocs was broken as of 7.9.0 (see #802). I rebuilt the whole thing from scratch — both local documentation builds and the ones on readthedocs. I think the new build is actually simpler than it was, which I hope will make it less sensitive to changes on ReadTheDocs or in dependencies.

What else changed?

• Fix bad conversion of array of empty strings to string. (#816)
• Move [[likely]] feature check back to compile time, to speed up configure.
• Support [[assume(...)]].
• Fix throw_null_conversion compile error when converting enums. (#807)
• Fix memory leak when failing to connect.
• Improve introductory docs for connections and transactions.
• No more autodetection of doxygen etc. Run it, or don't.
• Docs build now disabled by default; use --enable-documentation to change.

This is a pretty big release. As things stand, this looks to be the last release which supports C++17. The plan is to move on to libpqxx 8.0 next, which will require C++20 as a baseline.

Here's what's changed.

Assertion failure while streaming data

There was a bug that triggered an assertion failure when a row in a streaming query result ended in a null string.

The assertions in the code were just a little too strict. It amazes me that this took to long to surface. It has now been fixed.

New type aliases for binary data

We've been using std::basic_string<std::byte> and std::basic_string_view<std::byte> to represent binary data. But @tambry noted that compilers aren't actually required to support this! Worse, libc++18 had already marked it as deprecated.

The authors seem to have changed their minds about that, but Raul contributed a fix anyway. After all the problem may well pop up again.

And so, from libpqxx 7.9.0 onwards, to represent binary data coming from or going into the database, use the new type aliases pqxx::bytes (for a std::string of std::byte) and pqxx::bytes_view (for a std::string_view of std::byte).

If your environment supports the old types, these are just aliases for those, and nothing changes. But if it doesn't, then the aliases will refer to slightly different types that work around the problem. (The alternative definitions use a custom char_traits. The fine print in the C++ standard said that you need this, and that the library is not obligated to offer a generic definition of these traits for std::byte.)

Consistent library naming in CMake

Building CMake projects using libpqxx became a little easier. Thanks to @alexv-ds, you can now just use shared_link_libraries([...] libpqxx::pqxx).

The library name used to vary depending on whether you use find_package() or add_subdirectory() to add libpqxx to your project. Now it's just always libpqxx::pqxx.

Exception-related link errors

If you had a libpqxx built in C++17, and linked it into a project built using C++20, or vice versa, you'd get a lot of link errors about missing classes. They were related to exceptions, such as std::runtime_error.

Linking code built in one C++ version to code built in another is categorically dangerous. Please don't do it. There is no guarantee that it will work. Sadly though all package managers deal with this issue by sticking their heads in the sand.

It turns out that in practice the linking often worked, and various pre-built packaged versions of libpqxx shipped just one binary for all C++ versions. If you built your project in a different C++ version than was used to build libpqxx, two recent changes conspired to break your build:

1. For C++20 and up, I added source_location information to exceptions. If your compiler didn't support that, you just didn't get it — but it affected how an exception object was laid out in memory.
2. Instead of generating a header listing which C++ features were available when you built libpqxx, I just detected features at compile time. But that breaks down when the language changes between inbetween the libpqxx build and your own project's build!

The new release works around this using all-new code to generate a configuration header at build configuration time. The enw code is much more regular, and easier for me to extend and maintain. This should make it easier to add support for most new C++ features in the future. I also believe the build became just slightly faster.

Conversion from std::string_view to SQL string

Converting a std::string_view to an SQL string never actually worked. It wasn't a priority in part because pqxx::zview is likely to be much faster.

Still, this was an annoying irregularity and it has been fixed. You can now pass a std::vector<std::string_view> to a prepared statement, for example.

Expect future libpqxx versions to be a bit more liberal in allowing conversions of various view types. Which does mean that...

• sometimes the conversion may do a bit of extra work under the bonnet and it will be up to you to avoid this when performance is critical; and
• it will be your responsibility to ensure that the data underlying a view or span is still valid whenever you make use of its SQL conversion.

Parameterised versions of query() etc.

It has long bothered me that libpqxx has separate functions for executing a query, and for executing a query with parameters.

There are good reasons why you can't just pass any additional arguments to these functions and have them all converted into SQL parameters. It makes it easier to write code that doesn't mean what you might expect. It also complicates overloading, especially in a future where every query execution function also takes an implicit std::source_location to improve error reporting.

As of libpqxx 7.9.0 however you can now pass a pqxx::params when executing a query, and it will be unambiguously clear that it should be interpreted as a bundle of SQL parameters.

Streaming queries and C++20 pipelines

A streaming query can now act as a std::input_iterator. This removes an obstacle to building C++20 statement pipelines using streaming queries.

Clearer documentation for defining your own data types

The updated documentation makes it a bit easier to see how to go about defining SQL conversions for your own data types, so you can convert them between their SQL and C++ representations.

These conversions are particularly important when you want to pass them to parameterised or prepared statements.

Support for std::span as SQL arrays

This is still somewhat experimental, but libpqxx 8.0 will rely a lot more on std::span.

Thanks to @alexolog and @fallenworld1 you should now be able to pass any std::span (over a supported type of course) as a parameter to a prepared or parameterised statement, and it will automatically convert into an SQL array.

Support for PQfsize() and PQfmod()

You can now query a column's storage size and type modifier. This code was contributed by @TeamPlatform1.

These are only useful for low-level coders. Touch only if you know what you're doing.

Thanks

As you can see, various code changes have been contributed directly by libpqxx users. Others were requested in bug tickets. It would be a bit redundant for me to name them all here, but I am grateful: after all a good bug report is not so much a "customer complaint" as it is real-world feedback on what can be improved.

Further thanks go out to everyone who contributed, and not to forget — the tireless @tt4g and @KayEss who have stepped in to help time and again when people ran into problems.

Fixes two platform-specific build problems in 7.8.0:

Microsoft Visual Studio had been complaining loudly in C++20 mode when we included the deprecated header <ciso646>. But without that header, it turns out that Microsoft Visual Studio in C++17 mode can't compile libpqxx. So now we include the header only when compiling in C++17 mode.

The configure script built using autoconf did not support Apple's M1 and M2 ARM CPUs. Regenerating the script with a newer autoconf fixed that.

Welcome to libpqxx 7.8.0. Lots of goodies for you. Probably enough that I could have called it 8.0 — except libpqxx 8.0 is going to require C++20. For now you're still fine with C++17.

In 7.8 you get, among other things:

• Streaming large data sets now benchmarks faster than similar C/libpq code!
• New array class for easier parsing of SQL arrays.
• Deprecating stream_from. Use transaction_base::stream().
• Use array_parser only on comma-separated types, i.e. most of them. (#590)
• Bumping requirements versions: need postgres 10.
• Fix array_parser bug when parsing semicolon in an unquoted string.
• Make some zview constructors noexcept if string_view does it.
• Handle result status code for starting streaming replication. (#631)
• Faster text decoding and escaping in data streaming. (#601)
• Deprecate basic_fieldstream and fieldstream.
• Deprecate << operator inserting a field into an ostream.
• New string conversion fields: converts_to_string & converts_from_string.
• Support std::optional<std::string_view> etc. in stream_to. (#596)
• Remove support for single-quoted array/composite elements. No such thing!
• Work around build warning in MinGW: include winsock2.h before windows.h.
• If CMake can't find libpq, fall back to pkg-config. (#664)
• Work around spurious compile error on g++ pre-gcc-10. (#665)
• Include <pqxx/range> and <pqxx/time> headers in <pqxx/pqxx>. (#667)
• Don't use std::function as deleter for smart pointers.
• Work around gcc compile error with regex + address sanitizer + analyzers.
• Fix "double free" on exit when built as shared library on Debian. (#681)
• Stop including <ciso646>; should be built into compilers. (#680)
• New broken_connection exception subclass: protocol_violation. (#686)
• Retired unused blob_already_exists exception class. (#686)
• Support for PQinitOpenSSL(). (#678)
• Slightly more helpful error for unsupported conversions. (#695)
• Replace some C++ feature tests with C++20 feature macros.
• Support moving of stream_to. (#706)
• Incorporate source_location in exceptions.

There were some other small tweaks as well. If you implement the text conversions for your own types, there are two new fields that your string_traits specialisation: converts_to_string and converts_from_string. These are booleans that should say whether the string_traits class implements conversion, respectively, from the type to a string; and from a string to the type.

Enjoy this one! I'm sure somebody will find a problem, with so many changes, in which case we'll to a 7.8.1 soon.

= libpqxx 7.7.5: build fixes =

This patch update fixes the configure-based libpqxx install, and some of applications:

• make install after a configure-based build forgot to install some headers.
• The <pqxx/pqxx> header now also includes <pqxx/range> and <pqxx/time>.

This comes as a patch update, but it's a pretty big thing: libpqxx 7.7.4 comes with new, more convenient ways to query data.

In many cases you can simply forget about the pqxx::result class. Instead, use a transaction's "query" functions (query(), query1(), query01(), query_n(), query_value()) to execute an SQL query and go straight to the result data, converted into the C++ types of your choice. Or use for_query() to execute a query and on each row, invoke a callback you provide.

As a high-performance alternative, you can stream() your query. This works much like query(), except it starts iterating rows as soon as they come in (where query() waits for the full result first) and the rows come faster. Initial startup is a bit slower than with query(), but when you're expecting a lot of rows, it's likely to be faster and use less memory.

And then there's the classic "exec" functions (exec(), exec0(), exec1(), exec_n()). These return pqxx::result objects, apart from exec1() which returns a pqxx::row for convenience. Use these only when you expect no data rows, or when you need the result's metadata, such as the number of rows affected by an UPDATE statement.

I'd be interested to hear about your experiences! Do you still need pqxx::result at all in your application? Feel free to file a bug giving me some feedback.

This is mostly a bug-fix release. But also, it continues reorganising how libpqxx headers include each other. It's clearer and simpler now, it's easier to get right, and also, it seems to compile faster. Oh, and there's a new way to iterate over rows in a result or a streamed query.

Here's what else has changed:

• Fix up more damage done by auto-formatting.
• New result::for_each(): simple iteration and conversion of rows. (#528)
• Similarly, transaction_base::for_each() — stream a query and run a callback on each row.
• Add some missing headers in <pqxx/pqxx>. (#551)
• More strictness in header-pre.hxx/header-post.hxx checking.
• Disallow nesting of ignore-deprecated blocks.
• Deprecate exec functions' desc parameter.
• Fix placeholders documentation. (#557)
• Strip const and references from value_type. (#558)
• Get tests running on appveyor. (#560)
• Fix broken nonblocking connection on Windows. (#560)

This release may break some things for you:

1. If you included the wrong libpqxx headers directly into your own code, you will get an error message about it. Include only the <pqxx/something> headers yourself, not any of the .hxx ones.
2. The exec() query execution functions no longer accept a desc (description) argument. In C++20, we'll replace that with std::source_location so that we can automatically show the location in the source code where the failing query was issued. Or optionally, some different location that you pass explicitly.

Sorry, the 7.7.1 release was a dud. Rushed it, after bedtime, somebody talking at me. Try 7.7.2 instead.

Here's the 7.7.1 changes again:

The configure script no longer tries to tell your compiler what C++ language version it should compile. If you want a specific C++ version, you need to pass the right options to the compiler. Otherwise, you get your compiler's default C++ version.

Also:

• Fix digit_to_number not being found on some compilers. (#518, #519)
• In audit mode, define _FORTIFY_SOURCE to enable some extra checks.
• Make more functions constexpr. Nothing particularly useful though.
• Make more functions noexcept.
• Move constructor & move assignment for result.
• Deprecate set_variable/get_variable on transaction_base. (Design had unearthed warts in SQL variables, which were later fixed.)
• Set/get session variables on connection: set_session_var/get_var.
• Set/get local variables: execute SQL statements.
• When using select(), include <winsock2.h> if available. (#548)
• Fix those pointless MSVC warnings at last.
• Fix when cross-compiling using MinGW and CMake. (#548, #550)

I also reorganised the way libpqxx #includes its own headers. You may find that builds are a bit faster.

The main change in libpqxx 7.7.1 is that the configure script no longer tries to tell your compiler what C++ language version it should compile. If you want a specific C++ version, you need to pass the right options to the compiler. Otherwise, you get your compiler's default C++ version.

There are many other changes:

• Finally fix a long-standing silly warning in autogen.
• Fix digit_to_number not being found on some compilers. (#518, #519)
• In audit mode, define _FORTIFY_SOURCE to enable some extra checks.
• Make more functions constexpr. Nothing particularly useful though.
• Make more functions noexcept.
• Move constructor & move assignment for result.
• Support LGTM.com and Facebook "infer" static analysis.
• Deprecate set_variable/get_variable on transaction_base. (Design had unearthed warts in SQL variables, which were later fixed.)
• Set/get session variables on connection: set_session_var/get_var.
• Set/get local variables: execute SQL statements.
• When using select(), include <winsock2.h> if available. (#548)
• Library includes its own headers differently. Simpler, compiles faster.
• Fix those pointless MSVC warnings at last.
• Extract waiting functions into separate module.
• Fix when cross-compiling using MinGW and CMake. (#548, #550)

I also reorganised the way libpqxx #includes its own headers. You may find that builds are a bit faster.

It's going to be a big one. Believe me, I toyed with the idea of calling it 8.0. But I think it's not quite radical enough for that. I'm still hoping to raise the language baseline to C++20 for libpqxx 8.0. We'll see.

But here's what you get:

This version introduces a new class, connecting, for connecting to the database asynchronously. It's not native C++ async support, and it's not based on coroutines, but you can build coroutines on top of it. Kirit Sælensminde has built a sample project for this..

You can now convert SQL range values to and from C++ strings, using the new pqxx::range class.

There is now also support for converting between SQL timestamps (without time zones) and C++ std::chrono::year_month_day values. It's pretty limited, because there are lots of inherent problems with date/time support. For one, it only works with the ISO date format.

If you want fast access to a result's fields without having to go through row objects, the pqxx::result class now has a two-dimensional at() member that takes a row number and a column number. And since there's a proposal for permitting two-dimensional array indexing in C++23, there's a provisional two-dimensional operator[] as well. (However I can't test this yet, so it could be broken.)

The project now has a file requirements.json which lists the minimum major versions of compilers, C++, and PostgreSQL that you need for the current version of libpqxx. It's not going to be too detailed, because the more detail I put in there, the more is going to go wrong in maintenance. But at least it should be easy to see in the future which libpqxx versions require which C++ versions etc.

The Doxygen documentation now uses the (Doxygen-extended) Markdown format. That should make the documentation comments in the libpqxx headers a little more modern and easier to read.

Also, we now build docs in doc/html/, no longer in doc/html/Reference/.

There are also some changes that may affect you without exactly creating serious incompatibilities. More functions are [[nodiscard]]. Some member functions are now lvalue-qualified or rvalue-qualified. You can't call an lvalue-qualified function on an rvalue object — so I lvalue-qualified some functions that sane code would not call as the last operation on an object. I also rvalue-qualified one member function that effectively invalidates its object.

Finally, I promised you a lot of fixes:

• Fix stream_to for differing table/client encodings. (#473)
• Use [[likely]] & [[unlikely]] only in C++20, to silence warnings.
• Fix clang "not a const expression" error in Windows. (#472)
• Fix warnings about [[likely]] in if constexpr. (#475)
• Clearer error for ambiguous string conversion of char type. (#481)
• Statement name in prepare() error was for the wrong statement. (#488)
• Helper for implementing string traits: generic_to_buf.
• Work around broken std::filesystem::path in MinGW. (#498)
• Fix leak when getting client encoding fails. (#500)
• Re-enable pyflakes testing in tools/lint.
• Don't run clang-tidy by default. Compatibility issues with gcc options.
• Disable some std::filesystem features on Windows.
• Shut up stupid Visual Studio warnings.
• On gcc, mark rarely-used functions as "cold," to be optimised for size.
• Glyph scanning for GB18030 encoding was utterly broken. (#517)

Did I promise too much? Hope you all have a great 2022!

Support for the GB18030 encoding contained a bug that would probably have triggered an exception when trying to stream data or handle SQL arrays, composite values, streaming queries, or SQL range values.

See #517.

Support for the GB18030 encoding contained a bug that would probably have triggered an exception when trying to stream data or handle SQL arrays, composite values, streaming queries, or SQL range values.

See #517.

Support for the GB18030 encoding contained a bug that would probably have triggered an exception when trying to stream data or handle SQL arrays, composite values, streaming queries, or SQL range values.

See #517.

Support for the GB18030 encoding contained a bug that would probably have triggered an exception when trying to stream data or handle SQL arrays, composite values, streaming queries, or SQL range values.

See #517.

This bugfix update to the old 6.x series resolves a bug where executing a prepared statement would fail to activate the ongoing transaction.

That is, if the first thing you did in a transaction was execute a prepared statement, the actual transaction would not start at that point. The prepared statement would execute outside it.

Newer versions of libpqxx (as of 7.0) never had this problem because they start transactions as soon as you create them. If your environment supports C++17, please use those newer versions instead.

Bad news first: I just removed the ability to convert a string to std::basic_string_view<std::byte>. I'm sorry: this conversion should never have existed and it's not safe to use. See bug #463. It's not safe because it leaves no object to "own" the converted data. The view points into deallocated memory. If this breaks your code, use std::basic_string<std::byte> instead — so string, not string_view.

Now for the good news, of which there is a lot:

• Add C++20 concepts: binary, char_string, char_strings.
• In C++20: generalise binary strings to any contiguous range of std::byte.
• Mark zview as a view and as a borrowed range.
• Save a copy step on string fields in stream_to.
• New helper: pqxx::value_type<CONTAINER>.
• New helper: pqxx::binary_cast. (#450)
• Some escaping functions now have "into my buffer" variants.
• More generic escaping functions, supporting more types of binary.
• In C++20, accept generic columns list for stream_to. (#447)
• Renamed <pqxx/prepared_statement> to <pqxx/params>. (Old name still works for now, of course.)
• Deprecated dynamic_params in favour of params.
• pqxx::params::append_multi() now calls reserve() if possible.
• pqxx::params::size() is now noexcept (but sadly, ssize() is not — because std::ssize() isn't).
• Helper for generating parameter placeholders $1, $2, etc. (#443)
• Now requires support for C++14 [[deprecated]] attribute.
• Deprecated unesc_raw() in favour of unesc_bin() variants.
• Once unesc_raw() is gone, we'll support only the hex escape format for binary data.
• Work around broken thread_local in MinGW gcc < 11.1.
• pqxx::blob now supports std::filesystem::path.
• Fixed check against header/lib version mismatch: check_pqxx_version_7_6
• More complete documentation, of cursors in particular.

Finally, "result slicing" is now deprecated. It will go away in the future. Was anyone using this? It never felt complete or useful to me, and I haven't heard anyone mention it in at least a decade. Once we get rid of it, that'll shave a tiny bit of complexity out of your inner loops and make them a little more efficient. The main reason though is simplicity. Simpler code means fewer mistakes and surprises.

This is an emergency patch release. The recently added pqxx::blob::read() variant which takes a std::vector<std::byte> was utterly broken. It did not actually read any data from the blob.

This is now fixed, and properly tested.

This maintenance release of libpqxx works around a <thread> bug in MinGW; fixes some Visual C++ warnings; and makes the code a little easier to build out of the box by including config/compile in revision control.

But also, it deprecates more "legacy" representations of binary data, such as unsigned char *. We keep moving towards std::byte. If your compiler supports it, upcoming releases will move towards generic support for spans and ranges of binary data. If not, you'll need to represent binary data as std::basic_string<std::byte> and std::basic_string_view<std::byte>.

Eventually we'll raise the baseline C++ version to C++20, and at that point, it will be all ranges and spans.

There are now more ways to pass a BYTEA parameter to a query, and more of them get passed as binary data without intervening encode/decode pass. This last part makes it more efficient, but should not be visible to you otherwise.

When it comes to passing parameters to queries, some people wanted a way to build parameter lists on the fly. This was possible with the old parameter-passing API, but never actually supported. There is now a class for doing this, params, with some optimisation built in to prevent unnecessary copying of string_view and similar values.

"Blobs" (the new BYTEA API) have a new read() method on compilers which support C++20 "spans." It's based on std::span, and you can expect more use of spans soon.

Here's the more complete list:

• Now requires std::variant support! No longer works with gcc7.
• When implementing a string conversion, consider specialising param_format.
• Stop "aborting" nontransaction on closing. (#419)
• Silence an overzealous Visual C++ warning. (#418)
• Starting support for C++20 std::span.
• New blob::read() using std::span. (#429)
• New params class lets you build parameter lists incrementally. (#387)
• Pass std::vector<std::byte> params in binary format.
• Dropped legacy tests 31 and 49 to work around clang C++20 bug.
• Fixed stream_to test failure in non-English locales. (#440)
• Clarify transaction_base::stream documentation. (#423)
• Avoid <thread> on MinGW; it's broken there. (#336, #398, #424, #441)