crystal

RC3 is a small release with quite a lot of code churn. The reason for the churn is that the postgraphile module is now fully TypeScript, uses prettier for formatting, and has more lint rules enabled. I don't anticipate this causing any issues, but I'd rather not release v4 with such a large amount of code changed (even though it is mostly just formatting) without having the community test it first!

postgraphile-core does not have typings yet, so if you're importing into a TypeScript project feel free to grab some simple typings from here: https://github.com/graphile/postgraphile/blob/master/typings/postgraphile-core.d.ts

🙏 Thanks to my Patreon supporters for making this possible, and of course to the contributors mentioned below and everyone who has filed issues or improved the documentation! 🙏

New features:

• add --timeout CLI option thanks to @kronken
• TypeScript typings are now bundled thanks to @cdaringe
• PostGraphile now uses semicolons and prettier for code formatting
• getTypeAndIdentifiersFromNodeId was added to Build by @wieseljonas to enable replacing our default globally unique node identifier implementation with your own
• The new plugin interface has had a few changes and is maturing

Bug fixes:

• Error messages have been significantly improved in a couple places
• .postgraphilerc.js will no longer have errors swallowed silently
• An "uncaught promise rejection" error is now caught
• The 'money' type is now consistently parsed
• Fix an over-eager --no-ignore-rbac issue, tables with no permissions granted can now be exposed via security definer functions again (only affects people using --no-ignore-rbac)

Enjoy!

A plea: if you or your company want to see more documentation, more tests, and/or faster progress (or just want to ensure the maintainer doesn't burn out 😅) then please give generously to my Patreon or get in touch to discuss sponsorship. Ask your boss! Your financial support will enable me to take time off consulting and spend it working on the project, which would make me happy and should benefit you and your business too!

Huge thanks to my new and existing Patreon supporters; you've enabled me to spend more time on PostGraphile recently. 🙏

RC1 went down very well, no major bugs were discovered 🎉 However one of the major issues in PostGraphile has always been including too much in your generated GraphQL schema. This release helps to address this in two ways:

1. PostgreSQL extensions are now omitted by default which should significantly improve people's first impressions of PostGraphile. If you've followed the examples and best practices and use a schema other than public this should likely have no impact on you, but if you haven't then this is technically a breaking change. You may opt out via --include-extension-resources, but I am not aware of anyone using the functionality that has been removed, most likely it's just an annoyance.
2. Previously we ignored RBAC (i.e. GRANT statements) and exposed all operations and all tables/columns/functions in the schemas you tell us to. Well we still do this by default (because this would be a breaking change for a lot of people) but we now have the --no-ignore-rbac CLI flag (or ignoreRBAC: false library option) which will automatically remove fields from your schema that you're not granted access to. This goes as far as to remove fields from the create/update input objects separately based on INSERT/UPDATE column permission grants and honours the EXECUTE permissions on functions. I strongly recommend you give it a try, and let me know how you get on! This will be the default in version 5 of PostGraphile.

Another significant change that has been made is the internals of the plugin system now supports type modifiers (e.g. decimal(5, 2) was previously just seen as decimal), this means that PLUGIN AUTHORS AND USERS: some internals have changed, I've put backwards-compatible methods in to auto-port things to the new interface which should work fine, but you'll need to use the new interfaces to benefit from the new functionality (e.g. support for type modifiers).

A potentially breaking change for a very small number of applications is that if PostGraphile doesn't understand a type argument for a function then it will no longer expose it as a string, instead it will skip the function entirely. If this affects your application, please get in touch with me via electronic mail (benjie | graphile · org) or gitter and we should be able to knock up a plugin that adds back support.

Please let me know how you get on with RC2 via electronic mail (benjie | graphile · org) or gitter - your feedback is vital to the release process!

Full changes in RC2 vs RC1:

• Generated GraphQL schema can be much simpler!
  • introspects extensions, automatically ignores tables and functions that were installed by extensions - if you want these for some reason then you can disable this filtering via --include-extension-resources or includeExtensionResources: true
  • recommended: new --no-ignore-rbac CLI option (or ignoreRBAC: false library option) enables RBAC (GRANT/REVOKE) permission introspection and only exposes tables/fields/queries/mutations/functions that can be accessed by the connection string database user (or one of the roles it can change into)
• Koa support massively improved
  • no longer hangs when bodyParser() middleware is used
  • now is compatible with many more common Koa middlewares
  • now supports server-sent events, so Koa users get GraphiQL schema hot-replacement too!
  • no longer requires koa-connect compatibility layer (suggest you remove this!)
  • please file issues for any problems you face with Koa support
• PostGraphile's built in GraphiQL improved
  • auto-reinspects the GraphQL endpoint when the server-sent events connection re-establishes (e.g. when the PostGraphile server restarts)
  • handles changes to the GraphQL schema more gracefully - fixes a crash, and doesn't jump you all the way to root if it doesn't have to
• Functions (custom queries, mutations and computed columns)
  • New *BaseInput type added for tables, includes all columns, made nullable (like previous 'Patch' input type) - this is because the two existing types (Input and Patch) will be used for create and update mutations respectively, which now support RBAC, and your custom functions might require more fields. You won't find this in your schema unless you use the variant smart comments detailed below
  • New @omit category: 'base' for these *BaseInput types, so you can exclude columns from them also
  • Functions can now declare, via smart comment, which type of GraphQL table input they'd like to use via @arg0variant patch or @arg0variant base (replacing 0 with the zero-based index of the argument); if unspecified they will continue to use the standard *Input type, the one used for create* mutations. v5 might default to using *BaseInput for functions.
  • Functions no longer default to string if they can't find the correct input type for their arguments - they are now skipped instead. This should not affect many users.
• JWT support improved
  • bools are supported in JWTs again
  • fix an issue with audience verification
  • reminder that non-scalars in JWTs have changed - they're now handled via JSON.stringify(...) rather than String(...) - they were never officially supported - so if you were splitting a string coming from an array you're going to have to update that code.
• For plugin developers
  • Various PG helpers are now available on Build, so plugins can use them without having to declare graphile-build-pg as a dependency. Note that these methods are undocumented and thus may change in future versions without being a breaking change.
    • pgQueryFromResolveData
    • pgAddStartEndCursor
    • pgOmit
    • pgMakeProcField
    • pgViaTemporaryTable
    • pgParseIdentifier
  • pgGetGqlTypeByTypeId deprecated in favour of pgGetGqlTypeByTypeIdAndModifier - this allows us to support more complex types such as decimal(12,3) or geography(point, 4326) - see https://github.com/graphile/graphile-build/pull/233 for more
  • pgOmit now respects RBAC, so fields/types that don't have relevant permissions granted will no longer be exposed (disable via --ignore-rbac) - please update to using pgOmit from Build rather than importing omit from graphile-build-pg
  • attributes is now available directly on class introspection results - no need for a second lookup any more
  • graphile-build-pg no longer has dependencies on graphql - instead using the graphql passed from graphile-build
    • You can do the same in your plugins
  • Introspection now contains information about RBAC and extensions
  • you should not have to require graphql, graphile-build or graphile-build-pg in any plugins - if you do then please get in touch and we can figure out a better way
• Dev
  • Graphile tests now run against PG9.6 rather than PG9.4 (because wanted to use current_setting(text, boolean) in the tests, which was added in PG9.6) - PG9.4 should continue to work but note that officially we only support PG9.6+
  • Some light refactoring

Please support my Patreon: https://www.patreon.com/benjie Any amount helps, [Jimmy Wales voice] "if everyone using PostGraphile gave $10/mo right now..."

The time has finally come, the features are built, the bugs are squashed, the performance is incredible - it's Release Candidate time!

Please let me know on Twitter or on Gitter how you get on with this release (positive or negative) so we can move to a full release!

If you've not moved to v4 yet, you are strongly encouraged to do so - v3 will not be receiving any more updates, and v4 has some massive performance gains: it's up to 3x faster for simple queries, and for slightly more complex but still fairly typical queries it is 17x faster. Yes, that wasn't a typo - 1700% faster! The more complex the query, the greater the performance gains!

If you're coming from PostGraphQL v3, check out the migration guide here.

This release contains just one fix; and that fix was just a single-character! PostGraphile now correctly handles negative values in money fields thanks to @farant.

If you appreciate the work that goes into PostGraphile, please support the work on Patreon so we can get new features and fixes out faster!

Huge shout out to my Patreon sponsors - your support means the world to me!

I've massively increased the performance of PostGraphile - especially for trivially small queries. See the link for a detailed analysis, but TL;DR: on a Digital Ocean compute-optimised droplet with 8GB of RAM (running PostGraphile, PostgreSQL and the benchmarking software all through Docker) PostGraphile running in cluster mode over 4 vCPUs can serve 550 requests per second for the following fairly complex query (lists nested 3 levels deep, plus a few individual lookups in the mix), while maintaining sub-50ms 95th percentile latency:

query {
  allAlbumsList(condition: {artistId: 127}) {
    albumId
    title
    tracksByAlbumIdList {
      trackId
      name
      genreByGenreId {
        name
      }
    }
    artistByArtistId {
      albumsByArtistIdList {
        tracksByAlbumIdList {
          mediaTypeByMediaTypeId {
            name
          }
          genreByGenreId {
            name
          }
        }
      }
    }
  }
}

We also now support simple list collections as well as Relay connections; e.g. this:

{
  allAlbumsList(condition: {artistId: 127}) {
    id
    title
    tracksByAlbumIdList {
      id
      name
      genreByGenreId {
        name
      }
    }
  }
}

as well as this:

{
  allAlbums(condition: {artistId: 127}) {
    nodes {
      id
      title
      tracksByAlbumId {
        nodes {
          id
          name
          genreByGenreId {
            name
          }
        }
      }
    }
  }
}

Enable via --simple-collections both

This is hopefully the last release before we bump to 4.0.0 RC1! 🎉

New features:

• Massively improved performance
• Collections as lists (in addition to the current Relay connections) - enable via --simple-collections both
• Ability to disable the query log in the CLI version of PostGraphile

Refactoring:

• All graphile-build plugin hooks now accept the form (obj, build, context) => {...} rather than performing destructuring within the arguments - this should make the code slightly easier to read and more idiomatic - thanks to @mattbretl for this!

Fixes:

• GraphQL aliases longer than 63 characters no longer cause the system to throw errors (this might cause a small breaking change for plugin authors, but I think it's unlikely).

Queries that only request totalCount (and not nodes / edges) on function (not table) connections in beta 8 were broken. We now have tests for this so it cannot recur. Thanks to @enisdenjo for reporting.

Beta.7 was pulled because it introduced some regressions around certain rarer functions (e.g. computed column functions which return arrays of scalars, custom query functions which return sets of complex scalars such as interval).

This beta.8 fixes all the above, adds a load more tests to avoid regressions, and also fixes additional edge-cases relating to functions - particularly in the handling of mutation functions that return null table types.

Please refer to the beta.7 release notes for the other advances!

I'd welcome PRs which expand the tests to cover more situations to avoid regressions in future!

UPDATE: Do not use this release, it breaks handling of certain functions, use beta.8 instead.

There's a lot of stuff in this release, but the main thing by far is the ability to customise your generated GraphQL schema by adding comments to tables, columns, functions, relations, etc within your PostgreSQL database. These changes can be as simple as renaming something (via @name newNameHere) or can allow you to omit things from your GraphQL schema with quite a lot of granularity. For example, @omit update on a table would prevent the table from having an update-related functionality whilst maintaining queries, create and delete. @omit update on a column would prevent the column appearing in the Patch type, so it cannot be updated (but can still be created) via GraphQL.

This functionality has been documented by Jem here:

https://www.graphile.org/postgraphile/smart-comments/

As part of this I've completely overhauled how the inflector works; if you use the old inflector you should remove it ASAP - it will not be supported going forward. The new methodology allows the inflector to be customised easily with plugins, and moves it to graphile-build from graphile-build-pg so it can be used in all APIs - we no longer pass it in as an argument.

We also gain support for batched queries with this release - this functionality is experimental so you can try it out with --enable-query-batching. Unlike the rest of the system, this functionality has no tests - so do treat it as experimental, it may change in a future release.

There's loads of fixes and other minor fixes in this release too, and I've some exciting stuff on the horizon.

Huge thanks as ever to the 12 supporters on my Patreon who help make this work possible. And thanks also to the people who've been reporting issues, the bug reports have been particularly good recently - I appreciate it! 👍

Major features:

• @omit and @name (and ilk) smart comments enable you to remove and rename fields/types in your GraphQL schema
• The inflector is now pluggable (old interface is deprecated, stop using it ASAP!) and uses a slightly different API than before so it can pull information from comments/etc
• Experimental support for batched queries via --enable-query-batching

Major fixes:

• Solve issues where @skip / @include directives still resulted in the database being queried
• Solve issues with using variables in JSON subfields
• Solve issues where rows were returned as null if you only requested nodeId/id (node identifier) but not the primary key
• Solve issues handling procedures that return an array of custom types
• Remove babel-runtime and a number of other unnecessary dependencies

Other changes:

• Throttle watch re-introspection
• Introspection now pulls down constraint comments too (useful for smart comments)
• (For plugin authors) export parseIdentifier and resolveNode (experimental) functions
• (For plugin authors) add pgFieldIntrospection to GraphQLInputObjectType:fields:field scope
• (For plugin authors) enable extend to merge fields that have also been extended
• Solve stale comments from pg_upgrade issue faced by at least one user
• Added an experimental (undocumented) plugin interface, more on this soon...
• Refactoring
• Changed the dev script to make my life easier 😉

The handleErrors middleware option now allows you to write your own formatters for any GraphQLErrors thrown during GraphQL execution; this overrides extendedErrors and showErrorStack - you get full control over the errors, including the ability to add or remove errors should you so desire (not recommended!).

The handleErrors option is a function that receives 3 parameters:

1. the list of GraphQLErrors
2. the req object
3. the res object

It must return the array of errors to be added to the JSON HTTP response.

You could use this to add extra detail to the errors, handle translations based on request headers, set the response statusCode based on any errors, track errors via your error tracking service of choice, and much more.

This feature is thanks to @pyramation 🙏

This release also adds the X-GraphQL-Event-Stream header to /graphql responses when --watch mode is enabled, allowing tools to subscribe to the event stream for changes.

Since PostgreSQL 9.1 standard_conforming_strings has defaulted to on; this minor release fixes introspection if you have this configuration parameter set to off (not recommended!)

Thanks everyone for your feedback and support during the alpha phase - I'm proud to announce that PostGraphile beta is finally here!

The beta is the final testing phase before we produce the release candidates and ultimately the official version of v4. All of the anticipated breaking changes have been completed during the alpha phase, so the beta should be safe for general usage, including usage in production! Breaking changes from now on will have to be extremely well justified and should only be added in the case of show-stopping issues.

Sign up for announcements here: http://eepurl.com/cYwzUr

I've prepared a full v4 Feature Guide and a v3 → v4 Migration Guide - I encourage everyone to check them out - particularly people coming from PostGraphQL v3.

Aside: I take time off from my paid freelance work to advance the PostGraphile project - I'd love to work on it more, and if you'd like that too why not join my awesome Patreon supporters in contributing to the sustainability of this project!

For those of you who've been using the v4 alpha, the headline changes in this release are below. Please also note that postgraphql@next will not be receiving any more updates so anyone using that should upgrade to postgraphile ASAP.

Breaking changes since the last alpha:

• Requires Node v8.6+
• Internals have been renamed to postgraphile rather than postgraphql, so you may need to scan your codebase if you've been using the non-public interfaces (exports are aliased to their old names); there's also these related changes:
  • JWTs now have an audience of 'postgraphile' rather than 'postgraphql'
  • Watch schema is now postgraphile_watch rather than postgraphql_watch
  • DEBUG envvar now responds to postgraphile rather than postgraphql

New features over the latest alpha:

• --legacy-relations CLI flag for v3 compatibility (see docs)
• --legacy-json-uuid CLI flag for v3 compatibility (see docs)
• --no-server CLI flag (see docs)
• --cluster-workers highly experimental CLI flag (see docs)
• Much faster SQL building performance thanks to a lot of micro-optimisations in pg-sql2
• Config file support (experimental, undocumented)
• More JWT verification options
• Error message improvements
• Some GraphQL field descriptions have been improved

Docs: https://www.graphile.org/postgraphile/introduction/

• upgrade jsonwebtoken to work around security issue in dependency (if this breaks your app you need to submit an issue so we have better tests)
• duck-typing pg in more places
• add experimental support for .postgraphilerc.js
• if no connection string is specified, connect via standard envvar DATABASE_URL before falling back on postgresql envvars (⚠️ if you already have DATABASE_URL available in your environment and you don't use the connection string option then ensure this does what you expect! ⚠️ )
• add support for PGUSER and PGPASSWORD envvars
• doc tweaks

Thanks to everyone who has been patient about me merging their pull requests!

Anyone using postgraphql@next - please switch to using postgraphile as I will not be keeping postgraphql@next up to date soon.

This will be the LAST RELEASE OF postgraphql@next - future releases will all go out under the postgraphile moniker because managing both is absorbing too much time. If this is a problem for you, please get in touch. You should be able to switch out postgraphql@next for postgraphile without any issues.

v4 beta is not far away now, my list of remaining tasks has diminished radically:

https://trello.com/b/x4xS0bVG/postgraphile-v4-remaining-tasks

Changelog:

• Fix bigint/bigfloat precision loss
• Fix (remove) incorrect 'pagination' comments on mutation procedures
• Fix null cursors on edges on mutations
• Cursors on edges on mutations now support array of orderBy (potentially breaking change, but given it wasn't working I doubt you're relying on it...)
• Enums with asterisks in are better supported (potentially breaking change if your enums already have asterisks in)
• Better error messages when fields clash
• pg@7 support
• duck-type pg to avoid being told your config is invalid
• Warn when a schema is missing
• Autorename tables that end with input(s) or patch(s) so they don't cause conflicts (potentially breaking change if you have any tables named like this (which I deem unlikely))
• Performance: reduce redundancy in the SQL

Thanks for this release goes to all my Patreon supporters 🙏

This includes the last major feature required to replace v3 - caching the introspection results to disk (see #632). This is particularly important to anyone using PostGraphile on AWS Lambda or similar environments where startup time is at a premium.

• ✨ --write-cache <path> command line option enables you to write introspection results (and anything else that the various plugins, including your own, decide to cache) to a JSON file
• ✨ --read-cache <path> command line option reads the cache back from disk allowing plugins to skip certain time-consuming tasks like introspection

You should use --write-cache during the build phase, and then --read-cache when running in production. Please note that the cache does not get invalidated, so if you edit your database schema you will need to rebuild the cache otherwise things may get... interesting.

Use of this feature is not advised unless you understand the ramifications.

I really appreciate the support of the 7 patrons on my Patreon ❤️ Please consider joining them: https://www.patreon.com/benjie

We're so close to v4 beta now I can taste it! There's just one more breaking change I want to introduce - and that's #432, an improvement to create table bar (foo_id int primary key references foo, col text)-style one-to-one relations, so that you can query fooById(id: 7) { id barByFooId { col } } rather than what is currently fooById(id: 7) { id barsByFooId { edges { nodes { col } } } } - i.e. it removes a level of indirection. I hope to add an option to allow you to disable this if you need to restore backwards compatibility, the breaking change will be the change in default behaviour.

(Sidenote: if you're using PostGraphile on AWS Lambda, please allocate at least 512MB of memory or you may risk significantly larger response times according to @jmparsons on gitter - we have not optimised PostGraphile for memory usage yet).

The codemirror-graphql module now supports [email protected] which resolves the GraphiQL auto-complete issues that we've been experiencing; this release includes this updated dependency.

It's now possible to use --extended-errors to pass through even more keys from the pg error message, we advise you exercise caution when using this feature.

We now throw an error if you try to combine pgSettings function (or pgSettings object with a role key) with pgDefaultRole as this may lead to unexpected results.

Thanks to @explicitcall in #556, the pgSettings library option can now accept a function which returns settings to add to the PostgreSQL transaction. An example of using this might be something like:

app.use(
  postgraphql(process.env.DATABASE_URL, 'public', {
    pgSettings: req => ({
      'user.id': `${req.session.user_id}`,
    }),
  })
)

Be very careful when using this that you don't open yourself to cross-site request forgery (CSRF) security issues.

Thanks also to @vitaly-t for updating our version of pg-minify.

Version 4 is just around the corner with some significant memory usage and performance improvements, plugin support and more! For a list of breaking changes and to try it out, pop over to #506. Help testing it is greatly appreciated - please let us know whether or not it works for you (positive signals are important here too!).

This release adds --extended-errors where you may pass 'hint', 'detail', 'errcode' or all three (separated by commas) and errors produced will copy over these details from PostgreSQL, thanks to @zopf

We also fix our usage of pg-types, thanks to @vitaly-t

There were also documentation fixes thanks to @psyanite and @tim-field

Thanks everyone!

This release of v3 contains many fixes and improvements; plus it introduces the Docker image maintained by @angelosarto. There are a couple changes that may impact security depending on how you use PostGraphQL so please pay extra attention to the items in bold below.

ANNOUNCEMENT: v4 is just around the corner - it's a ground-up rebuild of the GraphQL schema generation to improve performance and support plugins to allow you to extend or modify the generated GraphQL schema to your hearts content! There's a few minor breaking changes but for most people it should be pretty much a drop-in replacement. I really need help testing it, and you can track progress on #506

Features

• GraphiQL now uses credentials: 'same-origin' (#436)
• Adds support for mutations that return void (#435)
• Set 401 status on expired token (#472)
• Warn if jwtSecret is set but jwtPgTypeIdentifier is not (#466 and #500)
• --jwt-role option to extract pgRole from arbitrary JWT path (#480)
• Generate Docker image (#496)

Fixes

• Documentation fixes (#423, #477, #497, #498, #503, and #507)
• Fixes processing of command line options --jwt-secret and --jwt-audiences (#434)
• Fixes building when path has spaces in (#441)
• Clarify computed column function criteria (#462)
• Document pgSettings option (#451)
• Don't throw an error when an Authorization header is present if no --jwt-secret specified (assume the application is handling it) (#437)
• Better performance for paginators by utilising CTEs to work around to_json performance issues (#396)
• JWT type names can now contain numbers 0-9, not just 1 and 2 (#511)

Contributors

I want to take this opportunity to thank @calebmer for all his work over the past couple years and for bringing this project to where it is today! He has moved on to working on the Flow team at Facebook, leaving some awfully big shoes to fill - as the new primary maintainer I hope that I can fill them with anywhere near the eloquence that Caleb had!

I'd also like to thank the (many) contributors of this release, including:

• @zorec
• @calebmer
• @nbushak
• @brysgo
• @benjie
• @kenips
• @florianherrengt
• @tim-field
• @chadfurman
• @mikeburgh
• @angelosarto
• @bradleyayers
• @ppKrauss
• @danielbuechele
• @gkchestertron

If you want to chat with us, join us on gitter: https://gitter.im/calebmer/postgraphql

Introduces the --jwt-audiences option, deprecates --secret in favour* of --jwt-secret and enhances the --help documentation. (#403)

Fixes to documentation. (#418 and #419)

* Apologies that you'll probably be dealing with English spelling from now on 😉