equinox

Released components: Equinox

Fix

• Equinox: LoadOption.AnyCachedValue, LoadOption.AllowStale prevent yielding of superseded value where overlapping call in flight #452
• Equinox: LoadOption.AnyCachedValue, LoadOption.AllowStale prevent incorrect TaskCanceledException outcome where overlapping call cancelled #452
• Equinox: LoadOption.AnyCachedValue, LoadOption.AllowStale correct to ensure optimal loading where first flight was in progress #452

Full Changelog: https://github.com/jet/equinox/compare/4.0.2...4.0.3

NOTE the version pinning changes under tag 4.0.1 remain unreleased (there will be explicit releases of the Equinox.*Store packages over time)

Released components: Equinox

Fix

• Equinox: LoadOption.AnyCachedValue, LoadOption.AllowStale caches and continually yields TaskCanceledException where request cancelled #451

Full Changelog: https://github.com/jet/equinox/compare/4.0.0...4.0.2

NOTE the version pinning changes under tag 4.0.1 remain unreleased (there will be explicit releases of the Equinox.*Store packages

This release is about 3 major versions in one (as evidenced by 1 alpha, 12 beta, 17 rc and multiple interim nuget releases!)

Major new features:

• DynamoStore: Full ground-up store implementation (including indexing/changefeed in the Propulsion.DynamoStore.* packages), using FSharp.AWS.DynamoDB 🙏 @ameier38 @epNickColeman @samritchie
• EventStoreDb: Replacement for the EventStore package (which uses the deprecated EventStore DB TCP/IP protocol) that uses their modern gRPC stack 🙏 @oskardudycz @thefringeninja @keppelerj
• MessageDb: Full ground-up store implementation for PostgreSQL based on Message DB with strong Open Telemetry support (that will eventually be implemented similarly across the board) 🙏 @nordfjord
• LoadMode.AllowStale: Support for bounded staleness (and limiting concurrent reads), enabling use for non-trivial read activity without having to adhere to strict CQRS separation and/or eventual consistency 🙏 @brihadish
• DeciderCore: C#-optimized API (arrays, Func, Task<T>, ValueTuple etc)
• Internal rewrite for performance, replacing usage of F# async with task, and AsyncSeq with [TaskSeq](https://github.com/fsprojects/fsharp.control.taskseq] 🙏 @abelbraaksma
• CosmosStore supports hydrating streams based on queries for Tip documents from across streams (i.e. you can now efficiently render a paged list of items based on search criteria against the snaphot's state in the Tip, in addition to the existing individual stream APIs) @brihadish @raghuatra
• CosmosStore internal implementation is now entirely based on System.Text.Json, with no internal Newtonsoft.Json usage) @ylibrach
• significant naming and structural influences based on the EquinoxJS project instigated by by
  Einar Norðfjörð 🙏 @nordfjord

API changes summary:

• All packages require net6.0 (i.e. it works seamlessly with NET 6.0 and and later, but there is no longer any netstandard2.0 support)
• Relies on FsCodec v3's
• FsCodec.StreamId (replacing the lower-level StreamName abstraction), streamlining how identity types get mapped to stream names
• Migrates from byte[] Event Bodies to System.ReadOnlyMemory<byte>
• Category base class enables significantly simpler wiring (see Factory/Store.Config structure in /samples and dotnet-templates
• Clearer Category APIs (e.g. name is passed to the constructor)
• Removal usage of F#-specific and/or otherwise deprecated types from the public interfaces:
• FSharpFunc->Func
• System.Tuple->System.ValueTuple
• list/seqs of events as returned from Decision functions are now arrays

Added

• Equinox: Decider.Transact, TransactEx overloads #325
• Equinox.LoadOption.RequireLeader: support for requesting a consistent read of a stream #341
• Equinox.LoadOption.AllowStale: Read mode that limits reads to a maximum of one retrieval per the defined time window #386
• Equinox.Category base class, with Decider and Stream helper modules #337
• Equinox.DeciderCore: C# friendly equivalent of Decider (i.e. Func and Task) #338
• Equinox.ISyncContext.StreamEventBytes: Exposes stored size of events in the stream (initial impl provides it for DynamoStore only) #326
• Equinox.Core.Batching: BatcherDictionary, BatcherCache to host concurrent Batchers #390
• Equinox.Core.Batching: Add limiter: SemaphoreSlim argument to extend linger phase #427
• CosmosStore.CosmosStoreConnector: Connect, ConnectAsync #421
• CosmosStore.Exceptions: Active patterns to simplify classification in the context of Propulsion handlers #416
• CosmosStore.Prometheus: Add rut tag to enable filtering/grouping by Read vs Write activity as per DynamoStore #321
• DynamoStore/DynamoStore.Prometheus: Implements the majority of the CosmosStore functionality via FSharp.AWS.DynamoDB #321
• EventStore: Revise test rig to target a Docker-hosted cluster #317
• EventStoreDb: As per EventStore module, but using the modern EventStore.Client.Grpc.Streams client #196
• MessageDb: Implements a message-db storage backend #339 with OpenTelemetry tracing and snapshotting support #348 🙏 @nordfjord
• eqx init --indexUnfolds cosmos: enables querying uncompressed unfolds (see shouldCompress) #434
• eqx query cosmos: Queries based on uncompressed unfolds (see eqx init -U) #434
• eqx query -o FILEPATH cosmos: Allows capture of raw JSON to a file #444
• eqx dump: -s flag is now optional
• eqx stats: -A flag to request all stats (equivalent to requesting -ESD) #424

Changed

• Change surface APIs that use Tuples and Options to struct equivalents. Some due to struct changes in FsCodec #82, and use task in hot paths #337
• Change surface APIs that use'event list or 'event seq to 'event[] #411
• Raise FSharp.Core req to 6.0.7, framework req to net6.0 #310 #337 #33 #411
• Replace AsyncSeq usage with FSharp.Control.TaskSeq v 0.4.0 #361 #391
• Equinox: Move Serilog dependency from Decider constructor to Category/Decider.forStream #337 #419
• Equinox: FsCodec.StreamId replaces usage of FsCodec.StreamName #353 #378 #419
• Equinox.ResolveOption: rename to LoadOption #308 #413
• Equinox.LoadOption: Rename AllowStale to AnyCachedValue #386
• Equinox.Decider: Replace 'event list with 'event[] #411
• Equinox.Decider: Replace maxAttempts with a default policy and an optional argument on Transact* APIs #337
• Equinox.Decider: rename Decider.TransactAsync, Decider.TransactExAsync to Transact #314
• Equinox.Core.AsyncBatchingGate: renamed to Batching.Batcher #390
• Equinox.Core.AsyncCacheCell: renamed to TaskCell #433
• Equinox.Core: Now a free-standing library that a) does not depend on Equinox b) is not depended on by the Stores (though CosmosStore inlines TaskCell) #420
• Stores: Change Event Body types, requiring FsCodec v 3.0.0, with EventBody types switching from byte[] to ReadOnlyMemory<byte> and/or JsonElement see FsCodec#75 #323
• Stores: *Category.Resolve: Replace Resolve(sn, ?ResolveOption, ?requestContext) with ?load = LoadOption parameter on all Transact and Query methods, and Decider.forStream/Decider.forRequest to convey request context #308
• Stores: *Category ctor: Add mandatory name argument, and Name property #410
• Stores: *Category ctor: Change fold to be a Func (no changes to F# code required) #421
• Stores: *Category ctor: Change caching to be last argument, to reflect that it is applied over the top #410
• Stores: *Category ctor: Change caching and access to be mandatory, adding NoCaching and Unoptimized modes to represent the former defaults #417
• CosmosStore: Require Microsoft.Azure.Cosmos v 3.35.4 #310
• CosmosStore: Switch to natively using JsonElement event bodies #305 🙏 @ylibrach
• CosmosStore: Switch to natively using System.Text.Json for serialization of all Microsoft.Azure.Cosmos round-trips #305 🙏 @ylibrach
• CosmosStore: Only log bytes when log level is Debug #305
• CosmosStore.AccessStrategy.MultiSnapshot,Custom: Change list and seq types to array #338
• CosmosStore.CosmosStoreCategory: Generalize compressUnfolds to shouldCompress predicate #436
• CosmosStore.CosmosStoreCategory.TryHydrateTip: Generates a Stream State Momento based on externally loaded unfold state #434
• CosmosStore.CosmosStoreCategory.TryLoad: Renders a 'state based on an Unfold #434
• CosmosStore.Core.Initialization.initAux: Replace hard-coded manual 400 RU with mode parameter #328 🙏 @brihadish
• CosmosStore.CosmosClientFactory: Moved to Core #430
• EventStore: Target EventStore.Client v 22.0.0; rename Connector -> EventStoreConnector #317
• Tool/samples/: switched to use Equinox.EventStoreDb #196

New Contributors

• @ameier38 was instrumental in the design and implementation of DynamoStore and many programming model advancements in V4
• @brihadish made their first contribution in https://github.com/jet/equinox/pull/328 and was key to the development of the concept of the bounded staleness AllowStale mode
• @epNickColeman made their first contribution in https://github.com/jet/equinox/pull/337 and drove #401; DynamoStore has correct ordering semantics thanks to major efforts on his part
• @keppelerj made their first contribution in https://github.com/jet/equinox/pull/371
• @nordfjord made their first contribution in https://github.com/jet/equinox/pull/339, added and polished the MessageDb store and was a huge impetus behind pretty much all positive changes in v4

Old contributors

• The DDD-CQRS-ES community are the reason this project exists and continues. Countless people influenced design decisions in this release directly and indirectly
• Multiple employers and colleagues over the course of the V4 development have provided boundless tolerance and support for always doing the right thing

Full Changelog: https://github.com/jet/equinox/compare/3.0.7...4.0.0

See CHANGELOG

Changed

• eqx dump/Equinox.Tool: Flip -P option to opt in to pretty printing for events https://github.com/jet/equinox/pull/313

Removed

• eqx dump/Equinox.Tool: Add -F option to opt out of pretty printing unfolds https://github.com/jet/equinox/pull/319
• eqx dump/Equinox.Tool: Show payload statistics https://github.com/jet/equinox/pull/323
• eqx dump/Equinox.Tool: Add -B option to prevent assuming UTF-8 bodies https://github.com/jet/equinox/pull/323

Fixed

• CosmosStore: Fixed mis-Indexing of Unfolds when >1 Unfold present https://github.com/jet/equinox/issues/322 https://github.com/jet/equinox/pull/324
• eqx dump/Equinox.Tool: Use System.Text.Json for pretty printing to handle null values correctly https://github.com/jet/equinox/pull/319

See CHANGELOG

Changed

• CosmosStore: Cleanup Microsoft.Azure.Cosmos calls #303

Removed

• Removed Grafana spec now that canonical is in dotnet-templates #304

Fixed

• CosmosStore: Fix fallback detection when all events pruned from Tip #306

See CHANGELOG

Added

• Expose .Log.PropertyTag Literals to enable log filtering #298
• Equinox.Tool: Add support for autoscaling throughput of Cosmos containers and databases #302 @belcher-rok

Fixed

• MemoryStore: Fixed incorrect Version computation for TransactEx post-State #296

See CHANGELOG

Fix

• CosmosStore: Fix dropping of events under reload with multiple writers, caching but not using snapshots #295 🙏 @ragiano215

See CHANGELOG

Added

• EventStore: Add customize hook to Connector #286

See CHANGELOG

Fixed/Changed

• Updated to MinVer 2.5.0, .NET SDK 5.0.200 in order to fix internal AssemblyVersion stamping 🙏 @mousaka

See CHANGELOG

Contributors: @enricosada @thinkbeforecoding @ylibrach

The emphasis of the V3 release is on the shift from Equinox.Cosmos (which was based on the closed source Microsoft.Azure.DocumentDb.Core package) to Equinox.CosmosStore (which is based on the Microsoft.Azure.Cosmos package).

CosmosStore notes

There are two key new features when one compares Equinox.CosmosStore with Equinox.Cosmos:

• events accumulate in the tip, reducing the document count in the store. In general this reduces overall RU consumption, but writes can cost more if you supply a large size limit when wiring up your CosmosStoreContext
• when creating a CosmosStoreClient, you can supply a secondary container, which enables one to have the proAchiver and proPruner templates
  1. archive to a warm store and
  2. prune from the hot store while
  3. the app falls back to the warm store if and only if it encounters events that have been migrated

NOTE: there are mild renaming changes when transitioning Equinox.Cosmos to Equinox.CosmosStore, but nothing that affects your implementation code.

Equinox.CosmosStore/Equinox.Cosmos stores are fully upward compatible.

NOTE Equinox.CosmosStore Events saved in Tip will not be visible to Equinox.Cosmos V2 users - if you need to be interoperable, you will need to set eventsInTip to 0

Added

• CosmosStore Replaced Equinox.Cosmos. See release notes for 3.0.0-beta1 through 3.0.0

Changed

• Equinox: Rename Equinox.Stream to Decider https://github.com/jet/equinox/pull/272

Removed

• Cosmos

See CHANGELOG

Added

• CosmosStore.CosmosStoreConnector: Wrapper for CosmosClientFactory enabling cleaner consumer code #292

Fixed

CosmosStore: Reinstate default connection mode as Direct per #281 #287

See CHANGELOG

Added

• Cosmos.Discovery.Endpoint: Extracts Uri for diagnostic purposes #284
• Cosmos.Prometheus.LogSink: Generalized app tag to arbitrary custom tags #287

Changed

• EventStore: Add customize hook to Connector #286
• EventStore: Pin EventStore.Client to [5.0.11,6) #286
• EventStore: Added SetAutoCompatibilityMode("auto") to default wireup #286

See CHANGELOG

Added

• Stream.TransactEx: extended signature, replacing TransactAsyncEx. Provides the ISyncContext both before and after the decide so pre-flight checks can be performed (as master #263) #277 🙏 @dharmaturtle for prompting it

Removed

• Stream.TransactAsyncEx (as master #263) #277

Fixed

• Cosmos.Prometheus: Correct namespace (was erroneously Equinox.CosmosStore.Prometheus) #271

See CHANGELOG

Added

• Cosmos.Prometheus: Prometheus integration package as master #267

Fixed

• MemoryStore: Serialize Committed events to guarantee consumption in event Index order #269 🙏 @fnipo
• Cosmos: Fix defaulting for compressUnfolds in C# #261

Added

• CachingStrategy.FixedTimeSpan: Support for fixed caching periods #255
• Cosmos: Support Serverless Account Mode in eqx init cosmos; default RU/s to 400 if unspecified #244 🙏 @OmnipotentOwl
• Cosmos: Added ability to turn off compression of Unfolds #249 🙏 @ylibrach
• Cosmos: Prune API to delete complete Batches from the head of a stream #233

Changed

• Cosmos: Reorganize Sync log message text, merge with Sync Conflict message #241
• Cosmos: Converge Stored Procedure Impl with tip-isa-batch impl from V3 (minor Request Charges cost reduction) #242
• Cosmos: Tweaked connecting log message

See CHANGELOG

Added

• Cosmos: Support Serverless Account Mode in eqx init; default RU/s to 400 if unspecified #244 🙏 @OmnipotentOwl
• Cosmos: Added ability to turn off compression of Unfolds #249 🙏 @ylibrach

Changed

• Cosmos: Reorganize Sync log message text, merge with Sync Conflict message #241
• Cosmos: Converge Stored Procedure Impl with tip-isa-batch impl from V3 (minor Request Charges cost reduction) #242

See CHANGELOG

• Cosmos: Added Prune API to delete events from the head of a stream #233
• Cosmos: Tweaked connecting log message

See CHANGELOG

• Core: Add AsyncBatchingGate #228
• EventStore: Pin EventStore.Client to [5.0.1,6) to avoid conflicting changes in EventStore.Client v 20.06 #223

See CHANGELOG

• MemoryStore: Add Committed event https://github.com/jet/equinox/pull/221

See CHANGELOG

🙏 @Kimserey for SqlStreamStore support bugfix

See CHANGELOG