datasette

Plugins and internals

• New plugin hook: filters_from_request(request, database, table, datasette), which runs on the table page and can be used to support new custom query string parameters that modify the SQL query. (#473)
• Added two additional methods for writing to the database: await db.execute_write_script(sql, block=True) and await db.execute_write_many(sql, params_seq, block=True). (#1570)
• The db.execute_write() internal method now defaults to blocking until the write operation has completed. Previously it defaulted to queuing the write and then continuing to run code while the write was in the queue. (#1579)
• Database write connections now execute the prepare_connection(conn, database, datasette) plugin hook. (#1564)
• The Datasette() constructor no longer requires the files= argument, and is now documented at Datasette class. (#1563)
• The tracing feature now traces write queries, not just read queries. (#1568)
• The query string variables exposed by request.args will now include blank strings for arguments such as foo in ?foo=&bar=1 rather than ignoring those parameters entirely. (#1551)

Faceting

• The number of unique values in a facet is now always displayed. Previously it was only displayed if the user specified ?_facet_size=max. (#1556)
• Facets of type date or array can now be configured in metadata.json, see Facets in metadata.json. Thanks, David Larlet. (#1552)
• New ?_nosuggest=1 parameter for table views, which disables facet suggestion. (#1557)
• Fixed bug where ?_facet_array=tags&_facet=tags would only display one of the two selected facets. (#625)

Other small fixes

• Made several performance improvements to the database schema introspection code that runs when Datasette first starts up. (#1555)
• Label columns detected for foreign keys are now case-insensitive, so Name or TITLE will be detected in the same way as name or title. (#1544)
• Upgraded Pluggy dependency to 1.0. (#1575)
• Now using Plausible analytics for the Datasette documentation.
• explain query plan is now allowed with varying amounts of whitespace in the query. (#1588)
• New CLI reference page showing the output of --help for each of the datasette sub-commands. This lead to several small improvements to the help copy. (#1594)
• Fixed bug where writable canned queries could not be used with custom templates. (#1547)
• Improved fix for a bug where columns with a underscore prefix could result in unnecessary hidden form fields. (#1527)

• Database write connections now execute the prepare_connection(conn, database, datasette) plugin hook. (#1564)
• The Datasette() constructor no longer requires the files= argument, and is now documented at Datasette class. (#1563)
• The tracing feature now traces write queries, not just read queries. (#1568)
• Added two methods for writing to the database: await db.execute_write_script(sql, block=False) and await db.execute_write_many(sql, params_seq, block=False). (#1570)
• Made several performance improvements to the database schema introspection code that runs when Datasette first starts up. (#1555)
• Fixed bug where writable canned queries could not be used with custom templates. (#1547)

• New plugin hook: filters_from_request(request, database, table, datasette), which runs on the table page and can be used to support new custom query string parameters that modify the SQL query. (#473)
• The number of unique values in a facet is now always displayed. Previously it was only displayed if the user specified ?_facet_size=max. (#1556)
• Fixed bug where ?_facet_array=tags&_facet=tags would only display one of the two selected facets. (#625)
• Facets of type date or array can now be configured in metadata.json, see Facets in metadata.json. Thanks, David Larlet. (#1552)
• New ?_nosuggest=1 parameter for table views, which disables facet suggestion. (#1557)
• Label columns detected for foreign keys are now case-insensitive, so Name or TITLE will be detected in the same way as name or title. (#1544)
• The query string variables exposed by request.args will now include blank strings for arguments such as foo in ?foo=&bar=1 rather than ignoring those parameters entirely. (#1551)

• Fixed bug where columns with a leading underscore could not be removed from the interactive filters list. (#1527)
• Fixed bug where columns with a leading underscore were not correctly linked to by the "Links from other tables" interface on the row page. (#1525)
• Upgraded dependencies aiofiles, black and janus.

• Fixed numerous bugs when running Datasette behind a proxy with a prefix URL path using the base_url setting. A live demo of this mode is now available at datasette-apache-proxy-demo.datasette.io/prefix/. (#1519, #838)
• ?column__arraycontains= and ?column__arraynotcontains= table parameters now also work against SQL views. (#448)
• ?_facet_array=column no longer returns incorrect counts if columns contain the same value more than once.

• Column names with a leading underscore now work correctly when used as a facet. (#1506)
• Applying ?_nocol= to a column no longer removes that column from the filtering interface. (#1503)
• Official Datasette Docker container now uses Debian Bullseye as the base image. (#1497)
• Datasette is four years old today! Here's the original release announcement from 2017.

• Fix compatibility with Python 3.10. (#1482)
• Documentation on how to use Named parameters with integer and floating point values. (#1496)

• Columns can now have associated metadata descriptions in metadata.json, see Column descriptions. (#942)
• New register_commands() plugin hook allows plugins to register additional Datasette CLI commands, e.g. datasette mycommand file.db. (#1449)
• Adding ?_facet_size=max to a table page now shows the number of unique values in each facet. (#1423)
• Upgraded dependency httpx 0.20 - the undocumented allow_redirects= parameter to datasette.client is now follow_redirects=, and defaults to False where it previously defaulted to True. (#1488)
• The --cors option now causes Datasette to return the Access-Control-Allow-Headers: Authorization header, in addition to Access-Control-Allow-Origin: *. (#1467)
• Code that figures out which named parameters a SQL query takes in order to display form fields for them is no longer confused by strings that contain colon characters. (#1421)
• Renamed --help-config option to --help-settings. (#1431)
• datasette.databases property is now a documented API. (#1443)
• The base.html template now wraps everything other than the <footer> in a <div class="not-footer"> element, to help with advanced CSS customization. (#1446)
• The render_cell() plugin hook can now return an awaitable function. This means the hook can execute SQL queries. (#1425)
• register_routes(datasette) plugin hook now accepts an optional datasette argument. (#1404)
• New hide_sql canned query option for defaulting to hiding the SQL quey used by a canned query, see Additional canned query options. (#1422)
• New --cpu option for datasette publish cloudrun. (#1420)
• If Rich is installed in the same virtual environment as Datasette, it will be used to provide enhanced display of error tracebacks on the console. (#1416)
• datasette.utils parse_metadata(content) function, used by the new datasette-remote-metadata plugin, is now a documented API. (#1405)
• Fixed bug where ?_next=x&_sort=rowid could throw an error. (#1470)
• Column cog menu no longer shows the option to facet by a column that is already selected by the default facets in metadata. (#1469)

• Columns can now have associated metadata descriptions in metadata.json, see Column descriptions. (#942)
• New register_commands() plugin hook allows plugins to register additional Datasette CLI commands, e.g. datasette mycommand file.db. (#1449)
• Adding ?_facet_size=max to a table page now shows the number of unique values in each facet. (#1423)
• Code that figures out which named parameters a SQL query takes in order to display form fields for them is no longer confused by strings that contain colon characters. (#1421)
• Renamed --help-config option to --help-settings. (#1431)
• datasette.databases property is now a documented API. (#1443)
• Datasette base template now wraps everything other than the <footer> in a <div class="not-footer"> element, to help with advanced CSS customization. (#1446)

• The render_cell() plugin hook can now return an awaitable function. This means the hook can execute SQL queries. (#1425)

• register_routes(datasette) plugin hook now accepts an optional datasette argument. (#1404)
• New hide_sql canned query option for defaulting to hiding the SQL quey used by a canned query, see Additional canned query options. (#1422)
• New --cpu option for datasette publish cloudrun. (#1420)
• If Rich is installed in the same virtual environment as Datasette, it will be used to provide enhanced display of error tracebacks on the console. (#1416)
• datasette.utils parse_metadata(content) function, used by the new datasette-remote-metadata plugin, is now a documented API. (#1405)

• Fix for an intermittent race condition caused by the refresh_schemas() internal function. (#1231)

• New datasette --uds /tmp/datasette.sock option for binding Datasette to a Unix domain socket, see proxy documentation. (#1388)
• "searchmode": "raw" table metadata option for defaulting a table to executing SQLite full-text search syntax without first escaping it, see Advanced SQLite search queries. (#1389)
• New plugin hook: get_metadata(datasette, key, database, table), for returning custom metadata for an instance, database or table. Thanks, Brandon Roberts! (#1384)
• New plugin hook: skip_csrf(datasette, scope), for opting out of CSRF protection based on the incoming request. (#1377)
• The menu_links(), table_actions() and database_actions() plugin hooks all gained a new optional request argument providing access to the current request. (#1371)
• Major performance improvement for Datasette faceting. (#1394)
• Improved documentation for Running Datasette behind a proxy to recommend using ProxyPreservehost On with Apache. (#1387)
• POST requests to endpoints that do not support that HTTP verb now return a 405 error.
• db.path can now be provided as a pathlib.Path object, useful when writing unit tests for plugins. Thanks, Chris Amico. (#1365)

• New plugin hook: skip_csrf(datasette, scope), for opting out of CSRF protection based on the incoming request. (#1377)
• POST requests to endpoints that do not support that HTTP verb now return a 405 error.
• db.path can now be provided as a pathlib.Path object, useful when writing unit tests for plugins. Thanks, Chris Amico. (#1365)

• The menu_links(), table_actions() and database_actions() plugin hooks all gained a new optional request argument providing access to the current request. (#1371)

• Fixed visual display glitch with global navigation menu. (#1367)
• No longer truncates the list of table columns displayed on the /database page. (#1364)

Warning: This release fixes a reflected cross-site scripting security hole with the ?_trace=1 feature. You should upgrade to this version, or to Datasette 0.56.1, as soon as possible. (#1360)

In addition to the security fix, this release includes ?_col= and ?_nocol= options for controlling which columns are displayed for a table, ?_facet_size= for increasing the number of facet results returned, re-display of your SQL query should an error occur and numerous bug fixes.

New features

• If an error occurs while executing a user-provided SQL query, that query is now re-displayed in an editable form along with the error message. (#619)
• New ?_col= and ?_nocol= parameters to show and hide columns in a table, plus an interface for hiding and showing columns in the column cog menu. (#615)
• A new ?_facet_size= parameter for customizing the number of facet results returned on a table or view page. (#1332)
• ?_facet_size=max sets that to the maximum, which defaults to 1,000 and is controlled by the the max_returned_rows setting. If facet results are truncated the ... at the bottom of the facet list now links to this parameter. (#1337)
• ?_nofacet=1 option to disable all facet calculations on a page, used as a performance optimization for CSV exports and ?_shape=array/object. (#1349, #263)
• ?_nocount=1 option to disable full query result counts. (#1353)
• ?_trace=1 debugging option is now controlled by the new trace_debug setting, which is turned off by default. (#1359)

Bug fixes and other improvements

• Custom pages now work correctly when combined with the base_url setting. (#1238)
• Fixed intermittent error displaying the index page when the user did not have permission to access one of the tables. Thanks, Guy Freeman. (#1305)
• Columns with the name "Link" are no longer incorrectly displayed in bold. (#1308)
• Fixed error caused by tables with a single quote in their names. (#1257)
• Updated dependencies: pytest-asyncio, Black, jinja2, aiofiles, click, and itsdangerous.
• The official Datasette Docker image now supports apt-get install. (#1320)
• The Heroku runtime used by datasette publish heroku is now python-3.8.10.

• Fix for a security hole: reflected cross-site scripting in the ?_trace=1 feature. #1360

• Docker image now allows apt-get install. #1320
• ?_col= and ?_nocol= options to show and hide columns in a table, plus UI for doing this in the column cog menu. #615
• New ?_facet_size=max option, linked to from the … on truncated facets. #1337
• Heroku runtime used by datasette publish heroku is now python-3.8.10.
• Multi-arch support for built Docker images using Buildx - thanks, Blair Drummand. #1319

Mainly dependency bumps, plus a new ?_facet_size= argument.

• Updated dependencies: pytest-asyncio, Black, jinja2, aiofiles, itsdangerous
• Fixed bug where columns called "Link" were incorrectly displayed in bold. (#1308)
• New ?_facet_size= argument for customizing the number of facet results returned on a page. (#1332)