JustSaying

A light-weight message bus on top of AWS services (SNS and SQS).

APACHE-2.0 License

Stars
42
Committers
67
View Code on GitHub View on X
Ecosystems: .NET, C#, Amazon Web Services

Bot releases are hidden (Show)

JustSaying - Version 7.1.1 Latest Release

Published by martincostello about 1 year ago

What's Changed

Full Changelog: https://github.com/justeattakeaway/JustSaying/compare/v7.1.0...v7.1.1

JustSaying - Version 7.1.0

Published by slang25 over 1 year ago

What's Changed

Features

Minor

New Contributors

Full Changelog: https://github.com/justeattakeaway/JustSaying/compare/v7.0.1...v7.1.0

JustSaying - Version 7.0.1

Published by slang25 over 1 year ago

What's Changed

Full Changelog: https://github.com/justeattakeaway/JustSaying/compare/v7.0.0...v7.0.1

JustSaying - Version 7.0.0

Published by slang25 about 2 years ago

It's been 4 years in the making, and we are very happy to announce the release of JustSaying 7 🎉

What's New ✨

◦ New fluent builder APIs 📖 docs

We've revamped the configuration APIs so bootstrapping your publishers and subscribers should feel a bit more natural, and uses consistent terminology.

◦ Integration with IServiceCollection 📖 docs

With the new JustSaying.Extensions.DependencyInjection.Microsoft package, it's now easy to add JustSaying to your services and get started.

◦ Subscription Groups 📖 docs

The message subscription pipeline has been completely overhauled and brings new capabilities. With this new functionality there is more control over how messages are fetched and processed.

◦ Middleware 📖 docs

Messaging processing has now been implemented as a pipeline composed of middleware. You can now add your own middlewares to this pipeline, and replace existing default middlewares.

◦ Infrastructure-less Mode

There is now the ability to connect directly to existing infrastructure using ARNs and Queue URLs where appropriate. There 2 new fluent APIs for using this; WithTopicArn and ForQueueUrl.

◦ Message Context 📖 docs

JustSaying now gives you access to the underlying SQS Message and attributes using the IMessageContextAccessor class (just like IHttpContextAccessor in ASP.NET Core)

...and much more!

# Contributors

A big thanks to all of our contributors that worked on this release ❤️

  • @martincostello
  • @gkinsman
  • @slang25
  • @AnthonySteele
  • @PeterKneale
  • @SimonCropp
  • @adammorr
  • @shaynevanasperen
  • @maisiesadler
  • @adamralph
  • @iancooper
  • @brainmurphy
  • @dominichickie
  • @maurofranchi
  • @josephwoodward
JustSaying - v7.0.0-beta.5

Published by gkinsman over 2 years ago

What's Changed

Full Changelog: https://github.com/justeat/JustSaying/compare/v7.0.0-beta.4...v7.0.0-beta.5

JustSaying - v7.0.0-beta.4

Published by gkinsman over 2 years ago

What's Changed

Full Changelog: https://github.com/justeat/JustSaying/compare/v7.0.0-beta.3...v7.0.0-beta.4

JustSaying - Version 7.0.0-beta.3

Published by gkinsman over 2 years ago

What's Changed

Adds support for late bound topic names, outlined in https://github.com/justeat/JustSaying/issues/1004 and https://github.com/justeat/JustSaying/issues/985. There is a new WithTopicName overload available on TopicPublicationBuilder that accepts a Func<Message, string> which is called at publish time to determine what topic to publish the message to.

Full Changelog: https://github.com/justeat/JustSaying/compare/v7.0.0-beta.2...v7.0.0-beta.3

JustSaying - Version 7.0.0-beta.2

Published by slang25 about 3 years ago

What's Changed

JustSaying 7 Beta 2 includes a new dispatcher pipeline to allow for better extensibility and control.

Full Changelog: https://github.com/justeat/JustSaying/compare/v7.0.0-beta.1...v7.0.0-beta.2

JustSaying - Version 7.0.0-beta.1

Published by slang25 over 3 years ago

First beta of JustSaying 7 🎉

JustSaying - Version 6.0.5

Published by martincostello about 4 years ago

Fix structured logging change from #757 causing the message to be logged incorrectly. (#759) Thanks @maurofranchi!

JustSaying - Version 6.0.4

Published by martincostello about 4 years ago

Added SNS request Id to log message when a message is published. (#757) Thanks @maurofranchi!

JustSaying - Version 6.0.3

Published by shaynevanasperen over 4 years ago

Fixed bug in MessageDispatcher in the case when a handler could not be found. Now logs an error and returns false, instead of just deleting the message.

JustSaying - Version 5.0.1

Published by brainmurphy about 5 years ago

Retention period for SQS queues should be 4 days #565.

JustSaying - Version 6.0.2

Published by brainmurphy about 5 years ago

Retention period for SQS queues should be 4 days #563.

JustSaying - Version 6.0.1

Published by AnthonySteele about 6 years ago

JustSaying - Version 6.0.0

Published by AnthonySteele about 6 years ago

This release consolidates gains made in JustSaying 5.0 version.

Version 5 put JustSaying on to the AWSSDK version 3 and NetStandard platform. Version 6 brings in several smaller changes to consolidate on that platform.

These changes will be source-compatible in some cases, but not in others.

The biggest breaking changes are expected to be:

  • Synchronous IHandler<T> was deprecated, is now removed. Use IHandlerAsync<T> instead.
  • The target platforms are now netstandard2.0 and net461.

This release requires the following IAM policy changes, which is a breaking change

  • sns:SetSubscriptionAttributes is now required on all Topics in order to support FilterPolicies (even if you aren't directly using this feature).

These new features are supported:

All changes:

A massive thank you to all of the following people who have contributed code and/or time to this release.

  • @adammorr
  • @AnthonySteele
  • @brainmurphy
  • @duncanbrown
  • @jaimalchohan
  • @JosephWoodward
  • @martincostello
  • @payman81
  • @shaynevanasperen
  • @slang25
  • @tonyharverson
JustSaying - version 6.0.0-beta4

Published by brainmurphy about 6 years ago

Bug fix to message filtering: Remove filter policies from queues if no policy is specified #387

JustSaying - version 6.0.0-beta3

Published by AnthonySteele about 6 years ago

Bring back support for NET461.
as this is not well-served by the Netstandard2.0 target.

JustSaying - Version 5.0.0

Published by slang25 over 6 years ago

JustSaying v5.0.0

This release has been a long time in the making, I want to say a huge thank you to everyone that contributed, and to everyone that was patient in waiting for this release.

The big changes in this release are:

Re-implemented on top of AWSSDK v3

In version 3 of the AWSSDK, Amazon changed the package names, and split them into smaller individual packages per service. They also removed synchronous calls for target frameworks other than the full .NET framework.

AWSSDK v3 is more modular that v2, there is a NuGet package for each service, and common code in AWSSDK.Core. It also targets .NET Standard 1.3, which makes it more portable and can reach other platforms, such as .NET Core 1.0 and up.

.NET Standard 1.6 Support

This means you can now consume this from any supporting platforms, including .NET Core 1.0 and up. We continue to target net451, so no one is left behind in this release.

Project and NuGet Package re-rationalization

In v4 there were 4 assemblies and one NuGet package. Most of these assemblies weren't logically decoupled and didn't really add anything. In the newer .NET project system, having multiple assemblies per package is fighting against the tools.

We now have 2 packages:
JustSaying
JustSaying.Models

Each containing just 1 assembly.
JustSaying.Models actually already exists, at version 2.0.0.x, but then disappeared from the solution after.
If you are using JustSaying v4 and any version of JustSaying.Models, then you could run into runtime issues, you will likely need to upgrade.

More async

We now have a PublishAsync method for example. See known issues for where we are not yet asynchronous.

Logging abstracted

In v4 of JustSaying and below, NLog was a hard dependency, this wasn't very friendly to people that don't use NLog, so we have abstracted this.
We settled on using Microsoft.Extensions.Logging.Abstractions rathen than reimplementing the wheel.
There are many providers today you can use, including NLog, Serilog and Microsoft provided ones such as Console logger etc...

Logging is a mandatory requirement for using JustSaying, if you want to turn off logging you should start your fluent configuration with:

.WithLogging(new LoggerFactory())

Message handling backoff strategies

We have added the ability to introduce a backoff strategy for message handling retrying.
There is a new interface IMessageBackoffStrategy, which looks like this:

public interface IMessageBackoffStrategy
{
    TimeSpan GetBackoffDuration(Message message, int approximateReceiveCount, Exception lastException = null);
}

The GetBackoffDuration method will be called if your message handler throws an exception, or returns false. You simply return the TimeSpan you want to wait for, and you can use the message, receive count, or last exception as factors for your backoff duration. Whatever your returned TimeSpan, JustSaying will set the message Visibility Timeout, which can be between 0 and 43200 seconds (12 hours), this ensures nothing else will attempt to dequeue the message for this duration.

Just pass the implementation during the subscription configuration like like:

var bus = CreateMeABus
            .WithLogging(loggerFactory)
            .InRegion(RegionEndpoint.EUWest1.SystemName)
            .WithMonitoring(_monitoring)
            .WithSqsTopicSubscriber()
            .IntoQueue("queuename")
            .ConfigureSubscriptionWith(cfg =>
                {
                    cfg.MessageBackoffStrategy = myFancyBackoffStrategy // <- NEW
                })
            .WithMessageHandler(_handler);

Ability to handle publish exceptions per publisher

You can now configure publishers and set the HandleException property with a delegate that can deal with the exception, and a boolean indicating if it has been handled or if it wants the exception to propagate.

Embedded source and embedded PDBs

The assembies now have the PDBs embedded, and to make debugging completely dependency free, the PDBs have the source embedded. Hurray!

All the fixes cherry-picked from v4

As this was such a big change, v5 was developed alongside v4, which contained some smaller (but important) changes, if you are upgrading from 4.0 then you will get the benefit from these too:

Discovering Topics more efficiently

When checking to see if an SNS Topic already exists, we were using the FindTopic AWSSDK method, which you might expect to make a single API call to return a single SNS Topic. Unfortunately it doesn't do this, within this method it calls ListTopics, then it picks out the topic in memory. For AWS accounts with a large number of topics, this became very slow.

We now don't make this check, as creating a topic is idempotent, this results in a rapid startup time for users under the above conditions.

Upgrade instructions

Let's take the following snippet that uses JustSaying v4, and let's update it to v5:

var bus = CreateMeABus
            .InRegion(RegionEndpoint.EUWest1.SystemName)
            .WithMonitoring(_monitoring)
            .ConfigurePublisherWith(c =>
                {
                    c.PublishFailureBackoffMilliseconds = 1;
                    c.PublishFailureReAttempts = 3;
                })
            .WithSnsMessagePublisher<GenericMessage>()
            .WithSqsTopicSubscriber()
            .IntoQueue("queuename")
            .ConfigureSubscriptionWith(cfg =>
                {
                    cfg.MessageRetentionSeconds = 60;
                    cfg.InstancePosition = 1;
                    cfg.OnError = _globalErrorHandler;
                })
            .WithMessageHandler(_handler);

After you update the JustSaying pacakge, be sure to tidy up any app.config assembly binding redirects that mention JustSaying.AwsTools and JustSaying.Messaging as these no longer exist, NuGet should do this for you.

The snippet above won't compile as we need to configure a logger before doing any further configuration:

// You might use a LoggerFactory throughout your application,
//   or you might be using your logging library abstraction, and this is just a bridge
var loggerFactory = new LoggerFactory();
loggerFactory.AddMyFavouriteLoggingLibrary(); // Example: .AddSerilog()

var bus = CreateMeABus
            .WithLogging(loggerFactory) // <- NEW
            .InRegion(RegionEndpoint.EUWest1.SystemName)
            ...

IAM permissions

In this release the AWS APIs that are called have changed slightly with regards to SNS, so it might be worth checking that you have the required IAM permissions set up.

Here are the actions requred if you are both publishing and subscribing.

sns:CreateTopic,
sns:ListSubscriptionsByTopic
sns:ListTopics
sns:Subscribe
sqs:CreateQueue
sqs:DeleteMessage
sqs:GetQueueAttributes
sqs:GetQueueUrl
sqs:ListQueues
sqs:SetQueueAttributes
sqs:ReceiveMessage

If you are just publishing then you can omit the SQS actions. If you are just subscribing then you still need the SNS actions (may change in the future).

Known issues

Synchronous initialization.

The fluent configuration is not lazy, and does asynchronous work (on non-full .NET AWSSDK is async only), but does so in a blocking way. The configuration is designed to be run once in your application in a context where blocking and thread usage should not be an issue.

We are aware that this isn't ideal, and have a branch where we are fixing it by adding an awaitable .BuildAsync() method, this is a reasonable breaking change so we want to take the opportunity to refactor and rationalise the fluent API design which will likely come in a future version.

Logging required up-front

Now that we have added configurable logging, and because the fluent APIs are not lazy, the first thing you have to provide us is a loggerFactory. Again this will likely change in a future version, where you would configure it at any point before awaiting .BuildAsync().

IHandlerResover changes

The Resolve method on IHandlerResolver previously used to be invoked a few times during initialization, then the returned IHandlerAsync instance was cached as singltons per subscription. This was counterintuitive, and was the cause of some confusion. We have simplified the behaviour and now Resolve will be called once per message. If you rely on this behaviour within JustSaying to give you the effect of singletons, you might need to change your resolver to ensure it is explicity shared across invocations.

Lots of dependencies when consuming from full .NET

We were careful to hand-pick the packages we depend on, rather than include just NETStandard.Library. However because we now depend on Microsoft.Extensions.Logging.Abstractions, version 1.1.2 of this will pull in NETStandard.Library 1.6.1, which will bring in "the whole world". This might be fixed by additionally targeting .NET Standard 2.0, which we may try for JustSaying 5.1

Contributions and thanks to:

  • @AnthonySteele
  • @pierskarsenbarg
  • @adammorr
  • @slang25
  • @andrewchaa
  • @JosephWoodward
  • @shaynevanasperen
  • @JonahAcquah
  • @brainmurphy
  • @martincostello
  • @Liewe
  • Tony Harverson
  • Mark England

... hope I haven't missed anyone

JustSaying -

Published by slang25 over 8 years ago

Bug fixes

Changelog:

  • #215 - Handler instance per request from IHandlerResolver
Badges
Extracted from project README
NuGet GitHub Discussions Build status codecov OpenSSF Scorecard