This released is lined up with 0.5.0 release of maddy mail server. go-imap-sql is still usable as a standalone go-imap backend but with some limitations.

Removed functionality

• go-imap-sql no longer implements any authentication - this is change for maddy 0.3 auth/storage split.

New features

• Add IMAP SORT support

• Add IMAP THREAD support

• Add IMAP NAMESPACE support

• Add ability to override mailbox/flags on per-user basis in delivery interface

Bug fixes

• Add missing counters update for EXPUNGE
• Fix X-Priority header field being missing from header cache
• Fix flags-only search returning duplicate IDs
• Fix Sender/Reply-To default values in returned envelope
• Fix handling of * seqset
• Do not assume clients specify date in APPEND command
• Fix creating index in mysql.

⚠️ This release includes schema changes that can't be applied to existing databases. Manual export-wipe-import is required.
⚠️ Minimal required Go version is increased to 1.13

This release includes changes accumulated over a long period. This release is lined up with maddy 0.1 release. go-imap-sql is developed for use in maddy mail server but remains usable as a standalone go-imap backend.

Removed features

• Code for storing message contents in table rows is removed, this has proven to be impractical (4c8996e)
  Messages contents now can be handled using only External Store interface.
• MySQL/MariaDB is no longer actively supported (45b437d).

Enchantments

• Usernames are now case-insensitive (3f1507c).
• Schema upgrade restrictions are removed, no more need to run imapsql-ctl with --allow-schema-upgrade (45cd989).
• Optional LZ4 and Zstd body compression is added (aa76135, d6bc61c).
• Additional interface optimized for efficient SMTP delivery is added. It was designed for use in maddy mail server (b4f87c7, 00be958, f53ec0a, 0d4a873).
• DB serialization failures are now reported as a separate driver-independent object (d7d4cb3f)
• pkg/errors library is replaced with Go 1.13 error inspection (19bcb2a)
• Message contents are not flushed to disk (fsync) before completing delivery or APPEND (6cd51e)
• Errors returned to clients are now logged (42bd4cfc)

RFC 3501 conformance

• INBOX is now created for each new user (4ab80a4)

imapsql-ctl features

• Message flags editing support is added (d5c43e4)

Performance

• Message creation uses Read Committed isolation with explicit locking. This allows concurrent writes without serialization failures (e09d6f8, dbd4222).
• Rudimentary \Recent flag support is replaced with stubs (f1109de).
• FirstUnseenSeqnum updates are not longer generated after a new message is added. This is turned out to be expensive on PostgreSQL and is not required by RFC 3501 (12981df).
• FirstUnseenSeqnum query no longer scans entire messages table (9244a53, 0114762, cac48d9).
• Message count is now cached in a separate column instead of being recalculated at each command that requires it (9244a53, 0114762).
• (SQLite tuning) WAL autocheckpoint interval is increased to 5000 pages (b3a28be).
• Compiled auto-generated flag modification queries are now properly cached (d81a059)
• MOVE/COPY performance is slightly improved by removing unnecessary intermediate operations (161bee8)

Bugfixes

• Body reader is closed earlier during search-scan. Previous behavior could lead to FD limit exhaustion when FS storage is used (379ca8e).
• imapsql-ctl: Fix missing error checking in connectToDB (fe0c354).
• Fix header added to BODY[TEXT] when body is fetched multiple times (e724fbd).

Misc

• Remove bodyLen=15 assignment added during testing (ddc595f).
• fsstore: Reenable blacklisted tests (bd90023).
• Add missing cleanBackend call to tests (01cd62d).

⚠️ Manual intervention required while upgrading databases due to the critical bug in previous versions.

Patch release containing several bug fixes. Note that some fixed bugs are critical, so it's best to update ASAP.

Add schema_version to existing databases

v0.2 and v0.3 had broken schema_version logic leading to absence of schema_version table in newly created databases. Basically, execute the following SQL statements on go-imap-sql DB and you will be fine.

CREATE TABLE schema_version (version INTEGER NOT NULL);
INSERT INTO schema_version VALUES (3);

Bugfixes

• Fix schema_version not being set correctly for new databases (0f4dd6345189f536bc1fb9704cd94e14c1baf2fe)
• Fix all messages in FETCH responses getting the header values parsed from body of the first message (783440414a5dcf80dc896e7d79c852d154a483f0)
• Fix duplicate IDs returned in SEARCH ALL responses (304076ad0396f371729d8455d8e5f35d0b3cca46)
• Use go-message/textproto directly, this should correct many issues involving encoded MIME headers (d1499eb7359473054fe3fd7f0e87db9c20207031)

This release is a small incremental improvement over 0.2, it does not introduce many new features, but rather cleans some things up and fixes some bugs.

Also, I think overall go-imap-sql is stable enough to be considered "beta" instead of "alpha".

New features

• go-imap-sql can now uses bcrypt instead of sha3-512 for passwords, sha3-512 can be used if selected via configuration (0bc84f7c979b2481648608346555b1e60da29a88, 5b7f6f4ca210059548bd5c4b31778c5719643235, d4e61b52f67a3c0f071a3153b7c66a22d9ec7243, f87c5fc8b63e81622ec803099f117ad56936e62a)
• Passwords can be set to null value to disable authentication for an account (6f8a9c09f8bceff2e9095e2569e5e1833f0f6142)

Improvements

• GetOrCreateUser is now atomic (6f8a9c09f8bceff2e9095e2569e5e1833f0f6142)
• Content-Type, Importance, List-Post header fields are added to static cache set now (1d8f57bc21de8a1abda49fb1975b39d4bcca4b29, 13313b2237151aa3a745fdcad477d21dbe4ba8cd)
• Errors happened during authentication are not hidden as "invalid credentials" now (0f172cf16b2121ed749cfec7cb56b9bc56893309)

Bugfixes

• SQLite3 options are not set in DSN (bd8b3b0410d20fbccc5337b5b84279a120a8573c)
  This makes them apply for all "connections" and fixes permanent database locking when it is opened by somebody.
• go-imap is updated to latest version (with emersion/go-imap#254) (11af4fd3d3ad2f2e7e1814abf0d8b81470790db0)
  This fixes header fields improperly matched during search.
• go-imap-sql was not saving body properly in some cases when ExternalStore is used, this is fixed now (b9864f1d75bc709e71cf48e8e3a4b0c8a24b23c3)

Interface changes

• Mailbox.UidNext is unexported (68033b7f372e77956da16cd698b348ba61ef4f10)
• Per-component version constants are removed (add0ce9efde8c4c0f3e334c71fee4679b8dd1e11)

imapsql-ctl

• All drivers are enabled by default now (48d17af76c079f324e84da2a493269e3fe39efbb)

⚠️ DB schema is incompatible with 0.1
⚠️ Backwards-incompatible interface changes present (due to go-imap update)

The main focus of this release is to improve performance to a reasonable level and to fix some conformance problems.

Repository rewrite

I'm really sorry for the inconvenience. I accidentally left imapsql-ctl binary in the repository even before 0.1 and I don't want repo to weight 18MB. So I decided to rewrite Git history, removing this file from all commits. All references (including 0.1 tag) are updated to point to new commits. go.sum checksum is invalidated. Only last commit is GPG-signed.

Performance improvements

• SearchMessages does not read body from DB unless it is necessary (9207da63a844a5a76cc03e4c59b413a576d228f8)
• Body structure and a subset of the header are statically cached to make ListMessages faster (037e62d01631902cfff41479ac9a0ffb70a8e2c3)
• SearchMessages takes shortcuts using denormalized data on simple search criterias (f432507d91a975daf6e924260ec9da9e110dca9d)
• (SQLite only) 16KB pages are used by default (ab50fa6a6b69a1d94ec566733cf4d7140050f81b)
• (SQLite only) Set of performance tuning options is added to Opts (ab50fa6a6b69a1d94ec566733cf4d7140050f81b)

External store interface

• Message bodies can now be stored in stores external to database (a1937ab429081edd89dd27782912b6c44f309ee6)
• FS directory can be used as external store (943fcaec482f6b9f42c182c83a934a71f31f4dff)

RFC 3501 compliance improvements

• The \Seen flag is set when the body is fetched without PEEK (0f434787af1296637b84c998cc1cdf8e17ff75f6)
• Used keywords are added to FLAGS and PERMANENTFLAGS (206e4e819ba457b4c550a0b80df54ed36e6ec066)

Interface changes

• imap subpackage is merged with root package (52508125ffec458484b64cc4a4af7826e986d6d8)
• imapsql.NewBackend is renamed to imapsql.New (1039e4491fc7d22665639b9308f67c61005fb324)
• Get*User functions are renamed (a43a1681d51eab7254291de25472b7cfa4f7afa5)
• Backend.DB and Backend.Opts are exported now (a1937ab429081edd89dd27782912b6c44f309ee6)

Schema upgrade

• Incremental schema upgrades are implemented. This should allow us to remove DB schema from compatibility requirements. Note that upgrades are one-way and a new database will likely be unreadable by older versions. Always make backups before any updates (ca9e1d56b24b5472f1da0801e56df8f18d425e77).
• This version supports schema upgrade from 0.1, but it will be removed before 1.0 release so you shouldn't rely on its existence (ca9e1d56b24b5472f1da0801e56df8f18d425e77).

Note to 0.1 users

go-imap-sql 0.1 does not create the version tag required for schema upgrade to work correctly. You need to execute the following SQL statements to fix it.

CREATE TABLE schema_version ( version INTEGER NOT NULL );
INSERT INTO schema_version VALUES (1);

Then you need to run any action with imapsql-ctl with --allow-schema-upgrade option. Server should not be running during it.

imapsql-ctl --driver ... --dsn ... --allow-schema-upgrade users list

RDBMS compatibility

• MySQL 5.7 support is removed (3d8b54eb25b65c14f42897e8ce75b50c153fea1a)

Dependencies

• go-message is updated to 0.10.3 (1b6ae988b63d99c879cd4efaba9097361aeca044, 5184394dbee56098ab8dc9247f91342864d0f46c)
• go-imap is updated to 1.0.0-beta.5 commit (1b6ae988b63d99c879cd4efaba9097361aeca044, 5d25a099180ef49fafdb281448347e08d06d6142)

Misc

• Blackbox tests are moved to separate repository (9e4dacd301ad8c640e7396296012f4d6d8a2a81e)

This is the basic implementation of a working go-imap backend with several IMAP extensions supported.

This release is a small step out of many on the road to the 1.0 release. I decided to release 0.1 as a base point of reference and allow people to actually rely on some frozen code base instead of always moving 'dev' branch.

Initially, the project was named go-sqlmail because I thought it was necessary to implement special support to handle SMTP delivery. Now it is renamed to go-imap-sql because all SMTP-related features can be implemented outside of this project.

• 1.0 development status
• 1.0 tasks board

Implemented features

• Almost RFC 3501-conforming implementation of go-imap's Backend interface
• Correct unilateral updates support
• IMAP MOVE extension support
• IMAP APPENDLIMIT extension support
• IMAP CHILDREN extension support
• Auxiliary interfaces for users management
• Good black-box tests coverage
• Support for three major SQL engines: SQLite, MySQL, and PostgreSQL.
• Helper utility for database management (imapsql-ctl)

Missing features / known problems / quirks

• Mailbox deletion is not handled correctly. Current behavior may confuse clients as it actually breaks RFC 3501.
• MySQL 5.7 support is poorly implemented and considered deprecated. It will be removed before 1.0 release.
• ListMessages uses suboptimal SQL queries.
• SearchMessage always iterates all messages in a mailbox.

Maddy

Currently, maddy uses go-imap-sql 0.1 with SQLite by default as both storage and authentication backend.