asynq

Changed

• Updated package version dependency for go-redis

Added

• PreEnqueueFunc, PostEnqueueFunc is added in Scheduler and deprecated EnqueueErrorHandler (PR: https://github.com/hibiken/asynq/pull/476)

Changed

• Removed error log when Scheduler failed to enqueue a task. Use PostEnqueueFunc to check for errors and task actions if needed.
• Changed log level from ERROR to WARNINING when Scheduler failed to record SchedulerEnqueueEvent.

This version adds Task Aggregation feature and includes a few improvements and fixes.

Task Aggregation

This is a new feature which allows you to enqueue multiple tasks successively, and have them passed to the Handler together rather than individually. The feature allows you to batch multiple successive operations into one, in order to save on costs, optimize caching, or batch notifications, for example. See the Wiki page for details.

Added

• Group option is introduced to enqueue task in a group.
• GroupAggregator and related types are introduced for task aggregation feature.
• GroupGracePeriod, GroupMaxSize, GroupMaxDelay, and GroupAggregator fields are added to Config.
• Inspector has new methods related to "aggregating tasks".
• Group field is added to TaskInfo.
• (CLI): group ls command is added
• (CLI): task ls supports listing aggregating tasks via --state=aggregating --group=<GROUP> flags
• Enable rediss url parsing support: Thanks to @eleboucher

Fixed

• Fixed overflow issue with 32-bit systems (For details, see https://github.com/hibiken/asynq/pull/426): Thanks to @assimon!

Fixed

• Fixed Redis version compatibility: Keep support for redis v4.0+ (See issue #401 for details).

This version improves worker crash recovery by introducing a concept of worker lease for an active task.
It also adds an optional field to Config to customize the context passed to the Handler.

Important Note:
Since this version changes the logic of crash recovery, update of the library version should be done with a clean restart of the process.
In other words, please make sure that the process running asynq.Server shutdown cleanly before restarting the process with this new version.

Added

• BaseContext is introduced in Config to specify callback hook to provide a base context from which Handler context is derived
• IsOrphaned field is added to TaskInfo to describe a task left in active state with no worker processing it.

Changed

• Server now recovers tasks with an expired lease. Recovered tasks are retried/archived with ErrLeaseExpired error.

Added

• PeriodicTaskManager is added. Prefer using this over Scheduler as it has better support for dynamic periodic tasks. See the wiki page for an example of using PeriodicTaskManager
• The asynq stats command now supports a --json option, making its output a JSON object
• Introduced new configuration for DelayedTaskCheckInterval. See godoc for more details.

This release includes changes to support Prometheus integration in the Web UI. The integration is optional, there shouldn't be any changes in performance or behavior of the library in this release.

Added

• Package x/metrics is added.
• Tool tools/metrics_exporter binary is added.
• ProcessedTotal and FailedTotal fields were added to QueueInfo struct.

This release includes a few additions and one fix.

Added

• Latency field is added to QueueInfo.
• EnqueueContext method is added to Client.

Fixed

• Fixed an error when user pass a duration less than 1s to Unique option

This release includes an introduction of task retention after successful processing. With Retention option, you can specify how long the task should be retained in the queue as a completed task.

Changed

• NewTask takes Option as variadic argument
• Bumped minimum supported go version to 1.14 (i.e. go1.14 or higher is required).

Added

• Retention option is added to allow user to specify task retention duration after completion.
• TaskID option is added to allow user to specify task ID.
• ErrTaskIDConflict sentinel error value is added.
• ResultWriter type is added and provided through Task.ResultWriter method.
• TaskInfo has new fields CompletedAt, Result and Retention.

Removed

• Client.SetDefaultOptions is removed. Use NewTask instead to pass default options for tasks.

This release fixes CLI import issue

Changed

• Updated github.com/go-redis/redis version to v8

Fixes

• Fixes tool import issue described in #325

This release includes one addition to the server config.

Added

IsFailure config option is added to Config to determine whether error returned from Handler counts as a failure.

This release includes one fix.

Fixed

• Scheduler methods are now thread-safe. It's now safe to call Register and Unregister concurrently.

This release includes minor change around task typename and critical performance fix for tooling (CLI, Web UI).

Changed

• Changed Queue function to not to convert the provided queue name to lowercase. Queue names are now case-sensitive.
• QueueInfo.MemoryUsage is now an approximate usage value.

Fixed

• Fixed latency issue around memory usage (see https://github.com/hibiken/asynq/issues/309)

This release includes one change.

Changed

• Queue function is updated to not to convert the provided queue name to lowercase.

This release includes a minor change and critical fix in the task recovering logic (tasks left in active state due to server crash).

Changed

• Changed to execute task recovering logic when server starts up; Previously it needed to wait for a minute for task recovering logic to exeucte.

Fixed

• Fixed task recovering logic to execute every minute

Asynq 0.18 includes major API changes and changes to the tooling (WebUI and CLI).
The inspeq sub package is removed, and all types and functions from the package are moved to asynq package.
The upgrade guide is available here

Package API changes

Task

• NewTask function takes array of bytes as payload.
• Task Type and Payload is accessed by a method call (previously these were fields on Task).

Server

• Server API has changed. Renamed Quiet to Stop. Renamed Stop to Shutdown. Note: As a result of this renaming, the behavior of Stop has changed. Please update the existing code to call Shutdown where it used to call Stop.

Scheduler

• Renamed Stop to Shutdown.

Client

• Client.Enqueue returns TaskInfo (Previously returned Result struct)

Inspector

• inspeq package is removed. All types and functions from the package is moved to asynq package.
• Inspector.RunTaskByKey is replaced with Inspector.RunTask
• Inspector.DeleteTaskByKey is replaced with Inspector.DeleteTask
• Inspector.ArchiveTaskByKey is replaced with Inspector.ArchiveTask
• WorkerInfo field names have changed.
• Inspector.CancelActiveTask is renamed to Inspector.CancelProcessing
• Inspector List methods (e.g. ListActiveTasks) returns slice of TaskInfo

CLI changes

• asynq task delete|run|archive commands takes take ID as well as the queue name (previously required task key)
  For example:

asynq task delete --queue=QUEUE --id=TASK_ID

Web UI (asynqmon) changes

• Payload now shows "non-printable bytes" if the payload bytes are not human readable (e.g. binary format)

Redis

• Requires redis v4.0+ for multiple field/value pair support
• Internal redis keys/values has changed (please see the migration guide)

This release includes one bug fix.

Fixed

• Free unique lock when task is deleted (https://github.com/hibiken/asynq/issues/275).

This release includes one bug fix.

Fixed

• Fixed internal RDB.memoryUsage method.

This release includes one minor update to RedisConnOpt to support redis connection timeouts.

Added

• DialTimeout, ReadTimeout, and WriteTimeout options are added to RedisConnOpt types.

Fixed

• Replaced KEYS command with SCAN as recommended by redis doc.