machinery

Machinery is an asynchronous task queue/job queue based on distributed message passing.

MPL-2.0 License

Stars
7.5K
Committers
132

Bot releases are hidden (Show)

machinery - Release v1.2.2

Published by RichardKnop over 6 years ago

This releases contains some bug fixes and a new feature:

  • merged support for opentracing (for example, with jaeger)
  • refactored reflection logic, added some improvements and fixed couple small bugs
machinery - Release v1.2.1

Published by RichardKnop over 6 years ago

This releases contains some bug fixes and a new feature:

  • fixed precision bug with large integers (caused by JSON encoding & decoding since Javascript objects only support floating point numbers, therefor integers get converted to floats and then back to integers when being transferred in JSON format)
  • a custom retry logic can now be implemented via returning a custom retry error (tasks.NewErrRetryTaskLater)
machinery - Release v1.2

Published by RichardKnop over 6 years ago

This release contains few bigger changes as well as several small improvements and bug fixes.

  • added support for slice task arguments
  • fixed several issues with AWS SQS broker
  • fixed Redis URL parsing
machinery - Fixed Config Loading Bug

Published by RichardKnop almost 7 years ago

Config loading would result in AMQP config being present even for non AMQP brokers because the default config contains it. This has been fixed so non AMQP configs are loaded correctly with AMQP config object being nil.

machinery - AWS SQS Improvements

Published by RichardKnop almost 7 years ago

Better documentation, tests and README for AWS SQS.

machinery - Added Support for AWS SQS Broker

Published by RichardKnop almost 7 years ago

Machinery now supports AWS SQS broker.

machinery - Fixed AdjustRoutingKey Bug With Redis Broker

Published by RichardKnop almost 7 years ago

AdjustRoutingKey has been fixed to check broker type and only rewrite routing key to binding key if the broker is AMQP and exchange is direct. Ignore this logic for Redis / other non AMQP brokers.

machinery - Ack Task After Processing

Published by RichardKnop almost 7 years ago

Tasks should be acked AFTER processing not before. This was fixed before but reverted by mistake.

See here: https://www.rabbitmq.com/confirms.html

Delivered tasks will be re-queued in case worker connection is lost. But otherwise ack should be sent after task is processed, not in advance.

machinery - Redis Delayed Task Gix

Published by RichardKnop almost 7 years ago

Delayed tasks with other than default queue as routing key did not work with Redis broker as once delayed task was fetched from the delayed_tasks queue, it was processed by the current worker process instead of being published to a proper queue (which might be different than current worker instance).

machinery - Config Creation Refactoring

Published by RichardKnop almost 7 years ago

Refactored and simplified config creation methods. Do not store config in internal variable anymore (was causing issues with multi config setups).

machinery - Add a non blocking LaunchAsync method to worker

Published by RichardKnop almost 7 years ago

Added a non blocking LaunchAsync method to worker struct as alternative to blocking Launch.

machinery - Improved Ctrl+C Handling

Published by RichardKnop almost 7 years ago

First Ctrl+C starts graceful shutdown (i.e. wait for running tasks to finish before shutting down). Second Ctrl+C will just abruptly end the program without waiting for tasks.

machinery - Wait For Tasks To Finish When Quitting

Published by RichardKnop almost 7 years ago

When there are running tasks and you call worker.Quit() method, the process will wait for all running tasks to be processed first before shutting down.

machinery - Updated Dependencies

Published by RichardKnop almost 7 years ago

Updated dep cli tool and all vendored dependencies.

machinery - Travis CI gometalinter bugfix

Published by RichardKnop almost 7 years ago

Fixed gometalinter issue on Travis CI so builds now pass successfully. Usage of native go tool vet is enforced now via gometalinter.json config file.

machinery - Return Arguments Are Now Appended To Success Callbacks

Published by RichardKnop almost 7 years ago

When a task has OnSuccess callback, the returned arguments are appended to the callback task instead of prepended.

machinery - MongoDB Backend Improvement

Published by RichardKnop about 7 years ago

Small MongoDB Backend Improvement to allow storing task results as bson.Json.

machinery - SendGroup Error Channel Size Fix

Published by RichardKnop about 7 years ago

Error channel in SendGroup could potentially have up to number of tasks * 2 size. Channel size increased to accomodate for this eventuality.

machinery - SendGroup Concurrency Fix

Published by RichardKnop about 7 years ago

Fixed a bug in SendGroup pooling logic which meant all tasks would be sent synchronously irrespective of the concurrency parameter.

machinery - Release v1.0.1

Published by RichardKnop about 7 years ago

When using AMQP as backend, x-expires parameter is set when declaring a queue in order to set/update task state. This ensures task state queues will eventually get cleanup up.