libpqxx

The official C++ client API for PostgreSQL.

BSD-3-CLAUSE License

Stars
1K
Committers
59

Bot releases are visible (Hide)

libpqxx - libpqxx 7.4.1: Fixing a build failure on macOS

Published by jtv over 3 years ago

This is a minor build fix. There are no other fixes, so if you have 7.4.0 and it works, this patch will make no difference to you.

But if 7,.4.0 failed to build on your system, this may well fix it.

libpqxx - libpqxx 7.4.0: "Named constructors," table paths, and more.

Published by jtv over 3 years ago

Named constructors

Some classes were getting too many constructors. Too much overloading makes for a programming minefield. So these classes are now growing factory functions, also known as named constructors.

For these classes, I would like applications to replace this kind of existing code:

pqxx::stream_to mystream(tx, "mytable");

with this new-style code:

auto mystream(pqxx::stream_to(tx, "mytable"));

Actually there's another change in that line; read on below.

Table paths

Sometimes you need to pass a table name to a libpqxx function. The function will often quote and escape that name internally. But what if the table name includes a schema name, and perhaps even a database name? If we quote the whole thing as one string, it will look to the database as a single weird name, with dots in it. What should happen is that libpqxx quotes each portion (database name, schema name, table name) separately, and puts bare dots between them.

To support that, there's a new way of describing table names: table_path, an alias for std::initializer_list<std::string_view>. Use this instead of strings to specify table names.

So instead of this:

auto mystream(pqxx::stream_to(tx, "mytable"));

...I would prefer you to write this:

auto mystream(pqxx::stream_to(tx, {"mytable"}));

The difference is that a table path is not just a name, it's a sequence of one, two, or three strings. (The last of those being the table name, of course.)

Which means that you can now also write:

auto mystream(pqxx::stream_to(tx, {"myschema", "mytable"}));

The rest

All major changes:

  • Work around Visual Studio 2017 bug with [[deprecated]]. (#405, #406)
  • Work around eternal Windows bug with max macro yet again. (#101)
  • Prepare for C++20 std::ssize().
  • Dropped test12, which didn't add much and tried to read null strings.
  • Support string conversion for std::monostate. (#409)
  • Consistent "named constructors" for stream_to and stream_from.
  • New table_path type.
  • New connection methods quote_table and quote_columns.
  • Lots of deprecated stream constructors.
  • pqxx::row::swap() now finally has the deprecated attribute.
  • Finally deprecated a bunch of field, row, and result constructors.
  • Exposed transaction_focus marker class.
libpqxx - libpqxx 7.3.1: New "large objects" API

Published by jtv almost 4 years ago

This new release replaces the largeobject class and friends with a single, much simpler blob class. Down with complexity! The new API has no streams; I don't think those stream classes really added much over a basic "write these n bytes" API.

The largeobject hierarchy now has the deprecated attribute, so your compiler may warn you when you use it. A lot of other things that were merely documented as deprecated now have that attribute as well.

In other news: the documentation now describes the concept of "transaction focus." This is how libpqxx enforces rules such as "you can't issue a query on a transaction while there's a pipeline open on that transaction," or "you can't have two table streams open on one transaction at once," or "you can't issue queries on a transaction that currently has an active subtransaction."

The errors themselves also got better: if you give your query a name or description —it's entirely optional— then an error message caused by that query can now actually mention that name. This is of no help when your code is running smoothly, but it can make debugging easier.

I'm also dipping a toe in the water of "named constructors." Some classes, particularly stream_from and the new blob, have too many constructors. Overloading can get a bit finicky, and the differences in meaning are not always very clear.

So for instance instead of just constructing a blob, for instance, and passing a "read-only please" argument, you do it like this:

    auto myblob = pqxx::blob::open_r(mytransaction, blob_id);

And similar for read-write or write-only. There is a new named constructor for stream_from as well, though only for the "stream from query" case. For the "stream from table" case, I'm still looking at the overloading possibilities for the list of columns.

libpqxx - libpqxx 7.3.0: no new features, but lots of changes

Published by jtv almost 4 years ago

Sorry, no shiny new features on this one. But a lot of std::string const & parameters are now either std::string_view or (if we're passing them on to the C-level library) pqxx::zview. Depending on how exactly you call those functions, it may speed up your code by eliminating some implicit string constructions and heap buffer allocations. Also some little compile fixes, of course.

Under the hood, though, it's a different story. I broke up the biggest inheritance hierarchy, which included almost all the major classes: the transaction classes, pipeline, streams, and so on. It is now two, much smaller, inheritance hierarchies. It rids us of the one case of virtual inheritance that the library had.

You may hit some minor compile problems with the changed parameter types. If you built your own transaction classes, you'll have to deal with some major changes. In the longer run we're going to drop the ability to plug your own transaction types into the hierarchy, and make more things final.

libpqxx - libpqxx 7.2.1: bug fixes, deprecations, and tweaks

Published by jtv almost 4 years ago

Here's a new patch release. Changes are:

  • Fix infinite loop in converting char * to string. (#377)
  • Deprecated namedclass.
  • Convert an entire row using row::as<type...>().
  • Internal rework of field::to() and field::as() functions.
  • Some more warning options in maintainer mode.
  • Removed the old, DocBook-based tutorial.
  • Fixed wrong query and SQLSTATE params to some exceptions. (#378)

Okay, okay, there's one nice new feature there. You can now convert an entire result row to client-side types at once. See row::as().

libpqxx - libpqxx 7.2.0: Another big bang release.

Published by jtv about 4 years ago

Many things have changed in this release:

  • You can now implicitly convert a const std::string & to zview.
  • Replaced some overloads for C strings and C++ strings with zview.
  • Deprecating binarystring. Use std::basic_string<std::byte> instead!
  • Array parser did not recognise escaping in unquoted values.
  • Document that string conversions assume non-null values.
  • Sketch out concepts-based PQconnectdbParams support. (#343)
  • Fixed infinite recursion when using std::optional in stream_to. (#364)
  • Catch floating-point negative overflow in check_cast, not underflow.
  • Bit more work on CMake build doc. (#318)
  • Experimental support basics for composite types. (#355)
  • Use stream_from without knowing the number of fields. (#357)
  • Global size_buffer function.
  • quote() now works for always-null types without conversions defined.
  • std::nullopt now converts to an SQL null.
  • New type trait: is_unquoted_safe.
  • Fixed mktemp invocation that broke on FreeBSD.

For a more complete list, see the NEWS file inside the codebase. Several more bugs were fixed, and there were performance improvements as well.

A few highlights in more detail:

String overloads. Many overloaded functions in libpqxx take "strings" of various kinds: there's C-style strings (char const *), C++ strings (std::string), there's std::string_view, and then there's pqxx::zview — which is a string_view where the creator promises that there's a terminating zero behind it. A bunch of these functions have been simplified and now just take zview. You can now implicitly convert a string to a zview too — but be careful to keep the string alive and unchanged while you may still use the zview!

Arrays. There was an important fix in the array parser, which previously did not support escape sequences in unquoted values. On the other hand, generating SQL arrays using to_string now makes fewer allocations, thanks to an optional new type trait.

Binary data. Another significant change is that binarystring is now deprecated. To work with binary values such as SQL's BYTEA type, represent them on the client side using the std::byte type: represent binary values as std::basic_string<std::byte> or std::basic_string_view<std::byte>. The string conversions know about these types, so you can use from_string() and to_string() to convert these types back and forth between their C++ and SQL forms.

Composite types. There is experimental support for SQL "composite types." You can now convert an incoming field of such types into a series of C++ values using the field's composite_to() method. Likewise there is a helper for representing a series of C++ values as an SQL composite-type object. The best way to use these is probably to write your own C++ type, and write custom string conversions for it. See datatypes.md for more about how to do this. The string conversions can call the existing libpqxx functions for convenience.

Enjoy!

libpqxx - libpqxx 6.4.7: Patch update to last pre-C++17 release

Published by jtv over 4 years ago

By request, here's an update to the last 6.x release. It fixes a link error where, when building libpqxx as a shared library, you might not have the nullptr_t specialisation of extract_value in stream_from.

This is in response to #340, but it may help others with compilers which don't support C++17.

libpqxx - libpqxx 7.1.2: Almost not a patch release

Published by jtv over 4 years ago

Lots of changes here, some important:

  • Document build in BUILDING-configure.md / BUILDING-cmake.md.
  • Work around silly clang warning in test_errorhandler.cxx.
  • Fix install error with some internal headers. (#322)
  • Fix "No object selected" error message in large objects. (#324)
  • If error has no SQLSTATE, throw broken_connection. (#280)
  • Fix argument order in encrypt_password. (#333, #334)
  • Fix underestimate of buffer size for to_string for floats. (#328)
libpqxx - libpqxx 7.1.1: some tweaks

Published by jtv over 4 years ago

A big release always reveals some things that need fixing. Here's a few build tweaks, and a fix for a missing file in the install.

libpqxx - libpqxx 7.1.0: A whole new way of querying data

Published by jtv over 4 years ago

Another big update.

The bad news:

  • Any string_traits<TYPE>::size_buffer() must now be noexcept.
  • There's an extra data member in nullness: always_null. You'll need to implement to give your type full functionality.

The good news: there's a new way of retrieving data from the database! It's faster, it's easier, it includes conversion to a type of your choice. Using transaction::stream, you can query rows from the database straight into a std::tuple, and so, into local variables:

for (auto const [id, name] : tx.stream<int, std::string_view>("SELECT id, name FROM item"))
    process(id, name);

And in other news...

  • stream_from now supports more or less arbitrary queries.
  • Instead of a tuple of fields, you can pass stream_to a container as well.
  • There is now to_buf support for string literals.
  • Streaming data is now more efficient.
  • The table name in stream_from is now escaped.
  • You can now "convert" strings to std::string_view. Mind your lifetimes!
  • A std::variant will now convert to string, if its member types do.
  • If a stream_from row fails to convert, you can no longer retry it.
  • from_string(field const &) now handles null values properly.
  • Obsolete Windows build docs are gone.
  • Added row::to(std::tuple<...>) — converts the whole row in one go.
  • Unified the test suites. We no longer need test/unit/unit_runner.
  • New helper: strip_t<...> to remove a type's constness and reference.
  • Replace some custom templating in CMakeFiles with CMake globs.
libpqxx - libpqxx 7.0.7: Build system tweaks

Published by jtv over 4 years ago

Here's a few improvements to both the "configure" build and the "CMake" build:

  • Fix broken --with-postgres-lib option in configure script (#311)
  • Should now build even if neither pkg-config nor pg_config is available.
  • CMake accepts PostgreSQL_ROOT, if it's a sufficiently recent version.
libpqxx - libpqxx 7.0.6: Yup, there's more to fix.

Published by jtv over 4 years ago

In the run-up to a 7.1 release, here's some more fixes:

  • Prefer pg_config over pkg-config for include path (#291).
  • Try to plod on if we don't know the PostgreSQL include path.
  • Fix error message when starting overlapping transactions and such (#303).
  • Fix potential crashes when converting invalid strings to floats (#307, #308).
libpqxx - libpqxx 7.0.5: minor fixes.

Published by jtv over 4 years ago

Updates libpqxx with a few more small fixes:

  • Compile fix for g++ 10: include <limits> (#292).
  • Cleaned up error-checking of PG results. (#280).
  • The esc() methods now also take std::string_view (#295).
libpqxx - libpxx 7.0.4: More fixes, more constructors, more operators.

Published by jtv over 4 years ago

Another small batch of changes:

  • Fix possible crash in connection::connection_string (#290).
  • Fix filtering of default values in connection::connection_string (#288).
  • Deprecate row::swap and public inheritance of iterators from row.
  • More copy/move/default constructors/assignments on result iterators.
  • More copy/move/default constructors/assignments on row iterators.
libpqxx - libpqxx 6.4.6: Important bug fix

Published by jtv over 4 years ago

This patch release fixes an important bug which could happen when reading large objects through the ilostream class. On systems where char is a signed type (which I believe includes most desktop systems), if the stream found any byte with value -1 at a buffer boundary, then it would mistake that byte for an end-of-file marker.

A similar fix was released in libpqxx 7.0.3.

libpqxx - libpqxx 7.0.3: Important large objects fix

Published by jtv over 4 years ago

This new version fixes an important bug which could happen when reading large objects through the ilostream class. On systems where char is a signed type (which I believe includes most desktop systems), if the stream found any byte with value -1 at a buffer boundary, then it would mistake that byte for an end-of-file marker.

Thanks to @tomlankhorst for finding and fixing this bug. A similar update for the 6.4 series is forthcoming.

There are also some other fixes: large-object streams open in binary mode. Some compile errors have been fixed (though the tests may not pass) on non-Unicode systems, and in some scenarios, lost database connections are now properly reported as broken_connection exceptions instead of sql_error exceptions. Unfortunately this does not cover all scenarios, so more work will be needed.

libpqxx - libpqxx 7.0.2: Small new features.

Published by jtv over 4 years ago

Here's a new libpqxx release. It's versioned like a bugfix update, but actually it adds a few small features:

  • New query_value method makes it easy to query a single value, and convert it to a given type in one go.
  • You can now iterate a stream_from stream. This may become very important.
  • More callable types qualify as transactors, thanks to std::invoke.

Why is iterating streams important? Because I'm currently testing a whole new way of querying data based on it. We're talking about streaming typed data straight from a database query.

If this pans out, expect a 7.1 release very soon. Early testing suggests that the new trick will be faster, easier to read, type-safe, and convenient. Of course there's a cost: your query may fail halfway through, after you've already begun processing data; and if it does, you'll just need to discard the entire connection and start over. Is that worth it? You will still have the choice to work either way.

libpqxx - libpqxx 7.0.1: exception hierarchy tweak

Published by jtv over 4 years ago

No matter how careful you are, there's always some detail you miss. I made the transaction_rollback a failure in the new exception hierarchy, when I meant to make it an sql_error. This exception and its sub-types happen when executing queries, and so they should be able to tell you the query, as well as report the "sqlstate" error code.

Technically this is an ABI break that should have happened in the 7.0.0 release. The constructor is different, which may break code. But only libpqxx should ever throw this exception, and perhaps some tests.

One way you may notice the difference is when catching both sql_error and transaction_rollback for the same try block. In that case, your compiler may warn you that the one is now derived from the other.

libpqxx - libpqxx 7.0: everything has changed!

Published by jtv over 4 years ago

Today we're releasing libpqxx 7.0. It's a radical change, and you may find that your software needs changes to compile.

For starters, C++17 is the minimum now. You'll find std::string_view everywhere, and I hope that you'll find libpqxx faster and more modern than before. If your compiler doesn't implement C++17 fully, that's probably okay — but you may have to pass an option such as -std=c++17.

If you implemented your own string_traits to support additional string conversions, you'll need major changes. The new system is faster, friendlier in some ways, but also a bit more complicated. There's a separate null_traits template, and conversions to and from char buffers.

Connections lost many of their advanced features. It's now a single, much lighter class. No inheritance hierarchy, no automatic connection reactivation. If you want to reconnect, create a new connection. Prepared statements and session variables are no longer cached, so when you define or undefine them, that goes straight to the server.

Invoking prepared or parameterised statements is now easier, thanks to template parameter packs.

The tablereader and tablewriter classes finally have worthy successors: stream_from and stream_to.

Transactors are much simpler: you create your transaction (and even your connection if you like), you commit it when done. A simple lambda can now be a complete transactor!

Exception classes have been rearranged. There is no longer a pqxx_exception base class, either.

Lots of enums have been moved out of classes, some have become enum classes.

Large objects now support 64-bit object sizes. Not for reads and writes though — writing multi-gigabyte chunks at a time is probably a bad idea.

Custom build files for Visual Studio or MinGW are gone. On systems that don't support configure, use CMake.

Then, the robusttransaction class no longer needs to create its own working table. It does require PostgreSQL 10 or better.

And that's not even all! The 7.0 development cycle took the better part of a year, and I'm sure there will still be things to improve. In the future I'm hoping to go C++20, with Concepts, async execution, and the many many other improvements we're expecting in the language. For now though, enjoy libpqxx 7.0 and C++17!

libpqxx - 6.3.4: Build fix for statement parameters

Published by jtv over 5 years ago

This is a minor update to 6.3, the previous stable release. It fixes the handling of const in smart-pointer-like arguments to prepared/parameterised statements.