asynq

Simple, reliable, and efficient distributed task queue in Go

MIT License

Stars
8.6K
Committers
41

Bot releases are hidden (Show)

asynq - v0.16.0

Published by hibiken over 3 years ago

This is a small release to add one feature.

Added

  • Unregister method is added to Scheduler type to remove an entry.
asynq - v0.15.0

Published by hibiken over 3 years ago

IMPORTATNT: All Inspector related code are moved to subpackage "github.com/hibiken/asynq/inspeq"

Changed

  • Inspector related code are moved to subpackage "github.com/hibken/asynq/inspeq".
  • RedisConnOpt interface has changed slightly. If you have been passing RedisClientOpt, RedisFailoverClientOpt, or RedisClusterClientOpt as a pointer,
    update your code to pass as a value.
  • ErrorMsg field in RetryTask and ArchivedTask was renamed to LastError.

Added

  • MaxRetry, Retried, LastError fields were added to all task types returned from Inspector.
  • MemoryUsage field was added to QueueStats.
  • DeleteAllPendingTasks, ArchiveAllPendingTasks were added to Inspector
  • DeleteTaskByKey and ArchiveTaskByKey now supports deleting/archiving PendingTask.
  • asynq CLI now supports deleting/archiving pending tasks.
asynq - v0.14.0

Published by hibiken almost 4 years ago

IMPORANT This release includes a breaking change, please install the latest version of CLI and run asynq migrate command.

Changed

  • Renamed DeadTask to ArchivedTask.
  • Renamed the operation Kill to Archive in Inpsector.
  • Print stack trace when Handler panics.
  • Includes a file name and a line number in the error message when recovering from a panic.

Added

  • DefaultRetryDelayFunc is now a public API, which can be used in the custom RetryDelayFunc.
  • SkipRetry error is added to be used as a return value from Handler.
  • Servers method is added to Inspector
  • CancelActiveTask method is added to Inspector.
  • ListSchedulerEnqueueEvents method is added to Inspector.
  • SchedulerEntries method is added to Inspector.
  • DeleteQueue method is added to Inspector.
asynq - v0.13.1

Published by hibiken almost 4 years ago

Fixed

  • Fixed shutdown timeout in processor. Shutdown timeout can be configured in Config.ShutdownTimeout and defaults to 8s if not specified.
asynq - v0.13.0

Published by hibiken about 4 years ago

Version 0.13.0 Adds Scheduler type for Periodic tasks

Added

  • Scheduler type is added to enable periodic tasks. See the godoc for its APIs and wiki for the getting-started guid

Changed

  • The interface Option has changed. See the godoc for the new interface. This would have no impact as long as you were using the exported functions to create Option (e.g. MaxRetry, Queue, etc)
asynq - v0.12.0

Published by hibiken about 4 years ago

Version 0.12.0 adds support for Redis Cluster and includes a number of breaking changes.

IMPORTANT: If you are upgrading from a previous version, please install the latest version of the CLI go get -u github.com/hibiken/asynq/tools/asynq and run asynq migrate command. No process should be writing to Redis while you run the migration command.

The semantics of queue have changed

Previously, we called tasks that are ready to be processed "Enqueued tasks", and other tasks that are scheduled to be processed in the future "Scheduled tasks", etc.
We changed the semantics of "Enqueue" slightly; All tasks that client pushes to Redis are Enqueued to a queue. Within a queue, tasks will transition from one state to another.
Possible task states are:

  • Pending: task is ready to be processed (previously called "Enqueued")
  • Active: tasks is currently being processed (previously called "InProgress")
  • Scheduled: task is scheduled to be processed in the future
  • Retry: task failed to be processed and will be retried again in the future
  • Dead: task has exhausted all of its retries and stored for manual inspection purpose

These semantics change is reflected in the new Inspector API and CLI commands.


Changed

Client

Use ProcessIn or ProcessAt option to schedule a task instead of EnqueueIn or EnqueueAt.

Previously v0.12.0
client.EnqueueAt(t, task) client.Enqueue(task, asynq.ProcessAt(t))
client.EnqueueIn(d, task) client.Enqueue(task, asynq.ProcessIn(d))

Inspector

All Inspector methods are scoped to a queue, and the methods take qname (string) as the first argument.
EnqueuedTask is renamed to PendingTask and its corresponding methods.
InProgressTask is renamed to ActiveTask and its corresponding methods.
Command "Enqueue" is replaced by the verb "Run" (e.g. EnqueueAllScheduledTasks --> RunAllScheduledTasks)

CLI

CLI commands are restructured to use subcommands. Commands are organized into a few management commands:
To view details on any command, use asynq help <command> <subcommand>.

  • asynq stats
  • asynq queue [ls inspect history rm pause unpause]
  • asynq task [ls cancel delete kill run delete-all kill-all run-all]
  • asynq server [ls]

Added

RedisConnOpt

  • RedisClusterClientOpt is added to connect to Redis Cluster.
  • Username field is added to all RedisConnOpt types in order to authenticate connection when Redis ACLs are used.

Client

  • ProcessIn(d time.Duration) Option and ProcessAt(t time.Time) Option are added to replace EnqueueIn and EnqueueAt functionality.

Inspector

  • Queues() ([]string, error) method is added to get all queue names.
  • ClusterKeySlot(qname string) (int64, error) method is added to get queue's hash slot in Redis cluster.
  • ClusterNodes(qname string) ([]ClusterNode, error) method is added to get a list of Redis cluster nodes for the given queue.
  • Close() error method is added to close connection with redis.

Handler

  • GetQueueName(ctx context.Context) (string, bool) helper is added to extract queue name from a context.
asynq - v0.11.0

Published by hibiken about 4 years ago

Added

  • Inspector type was added to monitor and mutate state of queues and tasks. See the godoc for details.

  • HealthCheckFunc and HealthCheckInterval fields were added to Config to allow user to specify a callback function to check for broker connection.

asynq - v0.10.0

Published by hibiken over 4 years ago

Added Features

  • Automatic recovery of tasks in the event of a worker crash
  • Automatic retry of tasks which exceeded its timeout/deadline

Version 0.10.0 includes the following API changes:

  • (*Client).Enqueue, (*Client).EnqueueIn, and (*Client).EnqueueAt has changed to return a *Result and error. A Result struct contains metadata about task that was enqueued (e.g. ID, Queue, etc).
  • ErrorHandler signature has changed to func(context.Context, *Task, error).

Version 0.10.0 includes the following semantics changes:

  • All tasks now require timeout or deadline. By default, timeout is set to 1800 seconds(30 mins) if neither of them are specified.
  • Tasks that exceed its deadline are automatically retried. In the previous versions, User provided Handler needed to explicitly return an error when ctx.Done channel is closed. In the new version, this is taken care of by the library. In order to avoid processing tasks when its deadline is exceeded, Handler should always check ctx.Done channel and stop processing when the channel is closed.

Other important changes:

  • Please upgrade to the new version of asynq CLI which is compatible with the new version of the library.
  • Encoding schema for messages has changed. Please install the latest CLI and run migrate command if you have tasks enqueued with the previous version of asynq.
asynq - v0.10.0.rc1

Published by hibiken over 4 years ago

Beta version for v0.10 includes the following API changes:

  • (*Client).Enqueue, (*Client).EnqueueIn, and (*Client).EnqueueAt has changed to return a *Result and error. A Result struct contains metadata about task that was enqueued (e.g. ID, Queue, etc).
  • ErrorHandler signature has changed to func(context.Context, *Task, error). Please use GetRetryCount(ctx) and/or GetMaxRetry(ctx) to get the count values that was part of the argument list in the previous versions.

Beta version for v0.10 includes the following semantics changes:

  • All tasks now require timeout or deadline. By default, timeout is set to 1800 seconds(i.e. 30 mins) if none of them are specified.
  • Tasks that exceed its deadline are automatically retried. In the previous versions, User provided Handler needed to explicitly return an error when ctx.Done channel is closed. In the new version, this is taken care of by the library. In order to avoid processing tasks when its deadline is exceeded, Handler should always check ctx.Done channel and stop processing when the channel is closed.

Other important changes:

  • Please upgrade to the new version of asynq CLI which is compatible with the new version of the library.
  • Encoding schema for messages has changed. Please install the latest CLI and run migrate command if you have tasks enqueued with the previous version of asynq.