docspell

Changelog • Documentation • Milestone

This release enables deleting multiple files at once of an item. It also changes how user settings are stored. Additionally several bugs in the ui and server have been fixed.

• Feature: Central user settings (#565): user settings have been stored in the browser but are now stored at the server. This means that all settings are now shared across all devices.
  The ui provides a migration button: goto user profile -> UI settings. If you have stored settings in the browser but not on the server, a button appears that pushes the settings to the server.
  Note: the button is only there if there are no settings at the server. So if you want to migrate, don't set the theme or click on other things that are persistet in the settings before doing the migration!
• Feature: Delete multiple attachemnts at once, thanks to @stefan-scheidewig (#626): multiple attachments on an item can be deleted with a single click
• Feature: Make consumedir-cleaner run on windows, thanks to @JaCoB1123 (#809)
• Bug: More work externalizing strings (#784, #760): many more strings have been externalized for being translated; also dates are now externalized, too
• Bug: Better anonymous upload page (#758): the upload page for anonymous users shouldn't show a form to provide any metadata
• Bug: Tag category color (#835): the input field to specify colors for tag categories didn't show the category name
• Bug: Search in names (#822): a bug in the webui sent a broken query to the server, making the "search in names" field unusable
• Bug: Fulltext only search broken (#823): the fulltext only search didn't only consult the solr index, but also the database, making it a lot slower and presenting the results not in the order returned by solr.
• Bug: Ui switches to logged in state on auth failure (#814)
• Bug: Broken search summary when tag has no category (#759)

Please see the cangelog for some more details. The milestone page lists all issues for this release.

Changelog • Documentation • Milestone

This release fixes some annoying bugs and prepares the web-ui to be translated into other languages. For actual translating them, I ask for your help. If you are interested, there is a detailed post about how to start with it in the docs.

• Refactor webui to prepare for localisation (#726)
• Add names to user defined tasks for better documentation (#712)
• Fixes some ui bugs:
  • scrollbar position (#722)
  • other minor ui related fixes (#746)
• Removed deprecated api endpoints, fixing #482
• Fixes bug where items are already shown in the ui, although still in processing (#719)
• Switch to github actions for ci (#748)
• Fixes a bug in the new query language (#754)
• Fix counters for categories in the search menu (#755)

Thanks to everyone showing interest in docspell and dedicating time by opening issues, testing and providing ideas!

Please see the cangelog for some more details. The milestone page lists all issues for this release.

Changelog • Documentation • Milestone

The main feature of this release gives a very flexible way of searching for documents, using a query.

• Add a query language to provide a flexible way for searching
  • the search form now is translated into a search query
  • allows to search for items not in a folder or not in a specific folder (#628, #379)
  • Allows for range searches in custom fields (#540)
  • And more! See the documentation
• Add a use attribute to all metadata, to be able to exclude them from suggestions (#659)
• Allow to submit items for reprocessing via the UI (#365)
• Add Latvian language (#679)
• Scrollbars are back! (#681, #677)
• The ds.sh script was changed to inform the processing jobs to also check for duplicates (#653)
• Docker setup now uses again a fixed hostname (#608)
• Moving the unit tests to MUnit (#672)
• Remove the old UI code (#636)
• Fix date extraction for English (#665)
• Fix bug when reading contacts from extracted text (#709)
• Fix bugs when reading mails (#691, #678)
• Fix a bug that wouldn't show an error message when entering bad characters in the register form (#663)
• Fixes a typo in the user settings menu (#654, thanks @ChristianKlass)

Thanks to everyone showing interest in docspell and dedicating time by opening issues, testing and providing ideas!

Please see the cangelog for some more details. The milestone page lists all issues for this release.

Changelog • Documentation • Milestone

This release comes with a completely new UI, including (finally!) a dark theme 🚀 🎊 . In addition many smaller features and bug fixes are included.

• New WebUI including a litght and dark theme.
  • The old ui can still be used, as a fallback. It is required to install a Browser Addon for this to add a request header. Check this for firefox or this for chromium. Then add the request header Docspell-Ui with value 1.
  • Experience on mobile devices is greatly improved
• The following UI issues are only included in the new UI:
  • foldable sections in search and multi-edit menu (#613, #527)
  • show current item in detail view (#369)
  • fixed some ui issues regarding processing logs (#363)
  • scrollbar fix (#600)
• Allow a person to be correspondent, concerning or both (#605)
• Add a short-name field to the organization (#560)
• Add a description field to the equipment (#633)
• Allow to specify a language for a source url (#651). This can be used to define upload urls per document language.
• Trim whitespace for certain fields (#539)
• A different docker entrypoint for the consumedir script was added that supports polling (thanks @JaCoB1123, #603, #624)
• Fix duplicate suggestions (#627)
• Fix reading mails with empty headers (#606)
• Fix suggesting person that doesn't belong to the suggested organization (#625)
• Cleanup registered nodes periodically (#618)

Please see the cangelog for some more details. The milestone page lists all issues for this release.

Changelog • Documentation • Milestone

This release comes with major improvements to the text analysis module. It is now much more configurable, has improved results and can learn tags from all categories. Additionally, more languages for document processing have been added and it's now easier to add more. Please open an issue if want more languages to be included.

• text analysis improvements (#263, #570)
  • docspell can now learn from all your tag categories
  • the detection for correspondents/concerned entities has been improved by using the classifier for this, too - all text analysis steps are now configurable that makes it possible to adapt it better to your data and machine.
  • The docs have been updated with some details here and here.
• more languages (#488)
  • Adds: Spanish, Italian, Portuguese, Czech, Dutch, Danish, Finnish, Norwegian, Swedish, Russian, Romanian
• scan mailbox change (#576)
  • The change from last version (#551) has been moved behind a flag in the "scan mailbox settings". Please review your scan mailbox tasks in your user settings.
  • The scan mailbox settings form view has been organized into tabs, as it grew too large for a single form.
• nix tools package fixed (#584)
  • If you are using docspell tools package for nix, it has now been fixed in that all scripts are available. They are now all prefixed by ds- (except the ds script)
• fix deleting organization (#578)
  • Due to the new relationship of a person to an organization, deleting an organization with references a person was not possible. This is now fixed.
• base url fix (#579)
  • The baseurl setting is optional, but when specified it was required to omit a trailing slash. This is now fixed in that it is always rendered without the trailing slash to the client, no matter what is in the config
• tag category case sensitive search fix (#568)
  • This was a bug introduced by the last release. When tag categories can now be spelled upper- or lower-case. In 0.18.0 you had to spell them lowercase, otherwise the search doesn't work.
• adds a workaround for mails that don't specify their used charset (#591)

Breaking Changes

• The joex configuration changed around text analysis. If you had some custom settings there, please review these wrt the new default config.
• When using the nix package manager: the tools package renamed the scripts to be better distinguishable, since they all end up in $PATH. They are now prefixed by ds-.
• The path of the consumedir script changed in the consumedir docker image
• The settings of the scan-mailbox task has been extended by another flag. If you were relying that all mails (even those excluded by a subject filter) where moved away, you need to check your scan-mailbox task settings.

Please see the cangelog for some more details. The milestone page lists all issues for this release.

Changelog • Documentation • Milestone

This release comes with many UI improvements and features as well as bug fixes. The screencasts on the website have been updated.

• Feature: Results summary and updated tag count (#496, #333)
  • A search summary can be displayed that shows the overall result count and to each custom field with a numeric type (number or money) small statistics like sum, average and max/min values. This is useful when you track your expenses on invoices or receipts.
  • This additional ui element can be enabled/disabled in your ui settings.
  • The result summary is now also used to update the tag counts in the search menu according to the current results.
• Feature: password reset (#376, docs)
  • Adds a new route for admins to reset the password of a user
  • Admin users are those with access to the config file, the endpoint requires to supply a secret from the config file.
  • A bash script is provided for more convenient access.
  • Note this also moves the re-create index endpoint behind the same secret! See below.
• Feature: custom fields clickable (#514)
  • The item detail view allows to click on tags to quickly find all tagged items. This now works for custom fields, too.
• Feature: scroll independently (#541)
  • The search menu can scroll now independent from the main area containing the item cards.
• Improvement: improve attachment selection (#396)
  • When selecting an attachment, it shows its preview to the name instead of the name only
• Improvement: wildcard search for custom date fields (#550)
  • Searching for custom field values allows to use a wildcard * at beginning or end. This is also enabled for date-fields.
• Improvement: joex memory (#509)
  • Joex currently requires a lot of memory to hold the NLP models. After idling for some time, which can be configured and defaults to 15 min, the NLP model cache is cleared. This reduces memory load and makes it possible for the JVM to give it back to the OS.
  • This is supposed to relieve memory consumption when idling only. However, whether it is reclaimed by the OS depends on the JVM and its settings. To observe it early, use the G1GC garbage collector. This is enabled by default for JDK11. So it is recommended to use JDK11 (which is used in the docker images).
• Improvement: allow scaling joex with docker-compose, thanks @bjeanes (#552)
  • This allows to easily start multiple joex containers via docker-compose
• Improvement: allow to connect with gmail via app specific passwords (#520)
  • Imap settings have been extended to be able to specify if a OAuth2 should be used or not.
  • Before, OAuth2 was the default when the server has advertised it. This has been changed now, which means you need to adapt your IMAP settings if you currently use OAuth2
• Fix: provide multiple possible date suggestions for English documents (#561)
• Fix: add missing language files to joex docker image (#525)
• Fix: fix a bug that occurs when processing is restarted (i.e. after a crash) (#530)
• Fix: fix a bug in the ui where the mail connection field was not correctly updated (#524)
• Fix: fix bug when importing mails with an applied filter (#551)

Breaking Changes

• Rest Server config:
  • If you specify the fulltext-search.recreate-key, you need to change your config. Delete it and use the secret now for the new setting admin-endpoint.secret.
• routes
  • The route to drop and recreate the fulltext search index has been moved. It is now at /admin/fts/reIndexAll. The secret must now provided as http header and not in the url.
• collective settings:
  • The imap settings have a new flag which indicates whether OAuth2 auth mechanism should be prefered. This is false by default. If you have used it with OAuth2 (like with gmail) you need either set this flag to true manually or use an application specific password.

Please see the cangelog for some more details. The milestone page lists all issues for this release.

Changelog • Documentation

I shouldn't have said that 0.17.0 would be the last release of this year… An unfortunate bug has made it into the previous release that makes the webapp near unusable. Therefore this release, containing only the fix for #508 (compared to 0.17.0).

I'm sorry for the inconvenience!

Please see the 0.17.0 release page for more info.

Changelog • Documentation

This release comes with some smaller features:

• Feature: Remember-Me – another cookie is used to provide a remember-me functionality. The cookie is checked against some value in the database, so an admin can always make all remember-me cookies invalid. (#435)
• Feature: Link persons to organizations. In the address book, a person can now be associated to an organzition. The dropdowns show a little hint for which organization a person belongs to. Also suggestions for persons are restricted to those of the organization if that has been associated before. (#375)
• Feature: Allow to filter on source names. The search form can now search by a source name. The new field can be hidden via ui settings (it must be activated for exising users). (#390)
• Feature: Customize the title and subtitle of the item card in the overview. You can now define patterns for the title and subtitle of a card. (#429)
• Feature: Export your data. A bash script has been added that goes through your items and downloads them all to disk including their metadata!). This can be used to periodically backup the data in docspell.
• Improvement: The webui has been improved in that the search bar and search form are unified regarding the text search. The two fields in the form, allowing to search in names and fulltext, have been combined into a single field just as the search bar. (#497)

This is the last release for 2020. I had hoped to put more into this, but this time of the year is always a busy one ;-). I want to thank you for your support and interest in this project and I wish you all a joyful Christmas time!

Changelog • Documentation

This release brings the "custom metadata fields" feature. It allows you to define custom fields and associate values to your items. Additionally there are some ui and other fixes and improvements.

ℹ️ Existing users must enable "Custom Fields" in their users settings in order to use this feature.

• Feature: Custom Fields – define custom metadata fields an set values for them on your items. For example, this can be used to track invoice numbers, pagination stamps etc. Fields can be defined per collective and carry a format (or type). (#41)
• Feature: The language has been added to the metadata of an upload request and therefore overrides the collective's default language. This means you can now set the document language with each document. (#350)
• Feature: Show the currently logged in user and the collective in the web app. (#329)
• Feature: Tag categories are presented as a dropdown, where you can choose an existing one or type a new one. (#331)
• Feature: The dropdown fields for a person have been changed in that the options are now restricted to the corresponding scope: the correspondent person only shows persons not marked as concerning and vice-versa. (#332)
• Feature: Add CC and BCC recipients to item mail (#481)
• The consumedir.sh scripts was improved:
  • log a warning for all subfolders that currently wouldn't work due to configuration problems (#386)
  • ignore hidden files on linux (starting with a dot .)
  • include the parameter skipDuplicates into the upload request when the -m option is present
• Fixes a bug that prevented detecting dates in january (#480, thanks @vanto!)
• Fixes updating search view after changes like deleting item in multi-edit mode or updating tags via drag-and-drop.

The list of issues is here.

There were changes to the http endpoints; please checkout the Changelog for details.

Changelog • Documentation

This release adds the "preview feature", where a thumbnail of the first page of each attachment can be shown in the overview. Additional there are some bugfixes and UI improvements.

ℹ️ Docspell will start creating preview images for all your files when starting. If you don't see all of them immediatly, wait some time until they have been all created. It starts with newest files, so you should see some images shortly.

ℹ️ Existing users must (if wanted) enable the preview feature in the user ui settings.

• Feature: create a thumbnail of every file. The scale factor can be configured in the config file. The user can choose in the ui, what size to display and whether to display it or not. (#327)
• Feature: Display the number of pages in the overview (#325)
• Feature: Extend the upload request to allow to specify a file filter and a list of tags (#346, #385):
  • file filter: this is a glob that selects files when an archive file (a zip or e-mail file) is uploaded
  • tag list: allows to specify a list of tags that are automatically set on the item that is being created
  • these two settings are added to the scan-mailbox form and the source-form
• Feature: Filter mails to import by subject (#384)
• Feature: Add quick links to item-detail and overview to jump to a specific search view; e.g. click on correspondent takes you to the overview with this correspondent selected in the search menu (#355)
• Improved css assets (#349)
  • Moved from Semantic-UI to Fomantic-UI, which is an actively maintained fork of the former
  • Removed the request to obtain a google font. Now you can use docspell without hassle in environments without internet connection
  • jquery could be dropped as a js dependency
• Improved ui for multi select mode when selecting items
• Fix a bug when creating new scan-mailbox settings (#382)
• Fix a build issue that resulted in missing scripts in the tools zip file.
• Fix a bug that added the .pdf extension twice. The filename can now be defined in the config file. (#397)

There were changes to the http endpoints and additional config values; please checkout the Changelog for details.

Changelog • Documentation

This release contains many bug fixes, thank you all so much for helping out! There is also a new feature that allows you to edit multiple items at once; and more scripts in tools section:

• Edit/delete multiple items at once (#253, #412)
• Show/hide side menus via ui settings (#351)
• Adds two more scripts to the tools/ section (thanks to @totti4ever):
  • one script to import data from paperless (#358, #359), and
  • a script to check clean a directory from files that are already in docspell (#403)

It is also worth noting, that Steve Streza created a shortcut for iOS that allows to conveniently upload documents from your iPhone/iPad!

Please check out the Changelog or the milestone for a list of bugs fixed.

Changelog • Documentation

This release contains bug fixes and brings an improved docker setup. Thanks to all for providing great feedback and code!

• Improvements to the docker setup: application can be build from any version and is automated by dockerhub. Thanks to @totti4ever!
  • This change required breaking changes in the docker-compose.yml file. Please update your docker-compose.yml to the new version.
  • The image tags changed:
    • the -latest is now upper case, -LATEST
    • tagged releases include the version prefixed with a v, like in -v0.13.0
    • there are new -SNAPSHOT images that are build from the current master branch. Please note that snapshot versions may not be compatible with each other!
• The date extraction tried to create invalid dates (#298)
• Fix content column for MariaDB (#297)
• Fix retrying processing of files: attached files were not correctly found and the duplicate check must not run
• Reset upload page on init (#294)
• Fixes regarding base-url setting and auth cookie (#308)
• Fixes in openapi spec (#338, #343)
• Fixed error messages for modal dialogs (tag/organization/person) (#341)

Changelog • Documentation

This release focuses again on the web ui. Thank you all so much who contributed great feedback!

• Improve startup experience wrt the base-url setting. Docspell now uses request headers to determine absolute urls if this setting is unchanged. (#250)
• Fixes scrolling issues when going from detail to list view and vice versa. (#254)
• Fix quick search component to keep search term between changes. (#261)
• Docspell now provides a valid manifest to be an installable "pwa".
• Fixes consumedir.sh script to work with whitespace in filenames. (#269)
• Fix hidden fields feature. Hidden fields are now resetted when clicking Confirm. (#256)
• Improve Save Name in item detail: The save button is removed and changes are saved while typing. (#255)
• Add keyboard shortcuts in item detail for navigating and confirm/unconfirm. (#225)
• Detect duplicate files server side. The processing task is now able to detect duplicate files to skip them if desired. The ui has a new checkbox when uploading. (#252)
• Improve view of many attachments to an item: The tab menu is replaced by a dropdown menu that allows to change attachments also on small screens and/or when there are many attachments.

This is a bugfix release. For existing installations that use the full-text search index, a required schema upgrade was forgotten. It is included in this release.

New installations are not affected and both versions, 0.11.0 and 0.11.1 are otherwise identical.

Changelog • Documentation

This release contains improvements for document processing.

• Improves recognizing correspondents/concerned entities for new documents. It makes now better use of the complete data (address book) when analyzing documents.
• Adds text classification feature based on the stanford-nlp classifier: Docspell can now learn from your existing tagged documents using machine learning algorithms (by looking at the extracted text). For new documents, a tag of a given category is suggested. The learning task can be scheduled periodically. You need to enable it in the Collective Settings page.
• Adds support for French in text extraction and text analysis. The joex artifact size increases by about 100M, because of the included model files.

See the Changelog for more information.

Changelog • Documentation

This is now the 10th release of docspell! This time the focus is in the web ui, especially the card- or list view. Some "extra" for the 10th release is the new Android App that makes it convenient to upload files from your android devices. Together with a scanner app, like for example the free OpenNoteScanner you can quickly scan and upload receipts and other documents.

Other notable changes include:

• Rework of the search menu. Tags and folders are now presented as list. Drag & drop can be used in the list view to toggle tags and move items in folders (drag an item onto a tag or folder).
• Search by tag categories (#203)
• The sources page shows buttons to easier copy the urls as well as a QR code for using it with the android app.
• The search results contain part of the item notes which can be displayed in the card and configured in ui settings (#186, #201)
• Item notes in the detail view are reworked and always shown (#186, #192)
• Allows to edit metadata in the detail few (in contrast to just adding new ones), the modal now only covers the menu and not the document view (#205)
• Allow to hide some fields in the search and edit menus (#195)
• Reprocess files. Items can be submitted to be reprocessed. Also a task is provided to convert all non-converted pdfs using ocrmypdf introduced in the last release (#206). This can be used to convert all items created before 0.9.0 into pdfs with text layer.

Have fun!

Changelog • Documentation

This release adds the "folder feature" and contains some bug fixes.

• New feature: folders. Users can create folders and put items into them. Folders can have members (users of the collective) and search results are restricted to items that are in no folder or in a folder where current user is a member. (see #21)
• Folders can be given to the upload request.
• Add ocrmypdf utility to convert pdf->pdf with ocr-ed text layer.
• Extract PDF metadata and use the keywords to search for tags and apply them to the item during processing. (See #175)
• Several bug fixes

Some more can be found in the Changelog.

Changelog • Documentation

This release is all about full-text search.

• This release contains the full-text search feature (see #69). Full-text search is optional and is backed by Apache SOLR, it requires a running solr instance. It is disabled by default.

  The docker setup provides this component out of the box and therefore has full-text search enabled in the config file. Other setups need to take care manually (but it's not hard, just install solr and create a core).

• The consumedir.sh script was extended to use the integration endpoint for uploading. This makes it more convenient to use and allows to upload for all collectives.

Some more can be found in the Changelog. There were additions to the config files and the rest api.

Changelog • Documentation

This release focuses on the WebUI when working with items and documents. Notable changes:

• Enable paging in the item list view. When scrolling down, a button at the end can load more items.
• The item list contains all tags of an item.
• Tag colors can be customized.
• Item metadata can be added directly in the item detail view. (See #138)
• Allow for multiple notify-due-items tasks.
• Adds a "basic search" feature that looks in item notes, name and all names of correspondents/concerned enities.
• Allow to change names of attachments.

There is more, please see the Changelog for details. There were additions to the config files and the rest api.

Changelog • Documentation

This release includes a few features, some improvements and bug fixes. Notable changes:

• Import e-mails from your mailbox via periodic polling. You can now setup one or more tasks to run periodically that fetch mails from your mailbox and imports them into docspell.
• Add files to existing items.
• Change order of attachments
• Frontpage has been overhauled; items are presented as “cards” and the search menu can be moved away.
• Fix webapp for mobile devices.

There is more, please see the Changelog for details. There were additions to the config files and the rest api.