cakephp-queue

Improvements

You can now use HTML5 progress bar (easier to style using CSS).
For this it is recommended to add the textual one from above as fallback, though:

$textProgressBar = $this->QueueProgress->progressBar($queuedJob, 18);
echo $this->QueueProgress->htmlProgressBar($queuedJob, $textProgressBar);

The textual bar will only be visible for older browsers that do not support the HTML5 tag.

The same goes for the timeout progress bars.

$textTimeoutProgressBar = $this->QueueProgress->timeoutProgressBar($queuedJob, 18);
echo $this->QueueProgress->htmlTimeoutProgressBar($queuedJob, $textTimeoutProgressBar);

For details see docs.

Bugfixes

Fixed PostgreSQL support for requestJob().
Travis is now fully active and passing for this DB type.

Bugfixes

Fixed progress display.

This is a BC breaking major for use with CakePHP 3.7+.

Please see Wiki for version map.

Main changes in API

• DB only queue (removed PID file solution and deprecations)
• All run() methods are now void. Exceptions must be used for the error case.

Improvements

• All configs are now defaulting to sane and secure values. Easier to start with the plugin.
• Provide visual and smart progress bars (uses the average of already run jobs if needed).
• Interface contracting also for add() method.
• No more autoloading configs from a file, you need to manually do that from your application.
• A provided QueueException can be used for exceptions without stack trace.
• Costs and Unique handling for servers and workers to avoid overloading.
• Add bake command to generate a Queue task as well as its test case.

This is a BC breaking major

Please see Wiki for version map.

Main changes in API

• DB only queue (removed PID file solution and deprecations)
• All run() methods are now void. Exceptions must be used for the error case.

Improvements

• All configs are now defaulting to sane and secure values. Easier to start with the plugin.
• Interface contracting also for add() method.
• No more autoloading configs from a file, you need to manually do that from your application.
• A provided QueueException can be used for exceptions without stack trace.
• Add bake command to generate a Queue task as well as its test case.

Bugfixes

• Fix logging to work
• Fix deprecated use of files for BC reasons

Improvements

• Added configurable return code check into Execute task. Added some better docs here, too.

Bugfixes

• Fixed error output on Execute task.

Improvements

Multiserver improvements

Using Queue.multiserver config you can now have the correct behavior per environment from both CLI and web backend.

Rerun

Added a re-run option for successfully run jobs on CLI
Just execute

bin/cake queue rerun MyTask {reference}

The optional reference can be an additional filter for specific jobs of this task.

Please note:
The API for reset() has slightly changed, but usually no one is directly using the result here.
It now returns the changed/touched rows (int) instead of boolean success.

Bugfixes

• Fixed backend URLs to be post-required for modifying data.

Bugfixes

• Fix local development and kill commands (Ctrl+C etc) to properly abort the queue worker process. Depending on your deployment process this could also be useful here to clean up the process list.

Improvements

• Add link to worker process if available. This link will be gone one the worker terminated.
• Server info will now be logged into error log data by default, as well

Improvements

Max workers per server

Added maxworkers config to set the max amount of workers per server. If not set or 0 it will behave as before, but with this > 0 it will assert that a cronjob cannot start more workers than currently running. It includes workers that died improperly to ensure this cannot bypass bottleneck situations where workers got marked as "to be stopped" but are still running some background script for minutes more.

Based on your memory and the jobs you are running you can estimate how many workers in parallel are the maximum you want to allow. This helps to prevent "Fatal error: Out of Memory" situations or other long-running jobs in parallel overloading your server (load >> 1).

Import/Export

It is now easier to export failed jobs and import them locally to re-run there for debugging.
You can import them as-is, or auto-resetted.

Bugfixes

• Fix pid to not be int casted.
• Validation requirePresence('workerkey') added for processes

Improvements

• Removed deprecations, CakePHP 3.7+ now.
• Easier job adding from CLI by default.
• Added invisible way (/data/ instead of /edit/ in backend) to modify data payload for trivial cases.
• Processes (running workers) can also now be terminated from the web backend.

Multi CLI Server Support

You can now run multiple worker servers:

• Tracking server name per worker now - make sure you have either env('SERVER_NAME') or gethostname() return a unique name per server instance.
• Added better deploy window handling using bin/cake queue end in the deployment script by shutting down active workers early. Either all (bin/cake queue end -q), or by server instance (bin/cake queue end server -q ).

To migrate make sure you execute this once all workers have been terminated (no active processes):

bin/cake migrations migrate -p Queue

Bugfixes

• Limit the error output on the overview page.
• Config Queue.exitwhennothingtodo is not required anymore

Improvements

• getOrFail() usage where required/needed.
• FrozenTime() usage for better timezone handling
• Prevent config loading from file if Configure key Queue.configLoaded is found.
• Include processes status in queue settings command to display currently running jobs.
• Colorful (green) success messages in CLI
• Nullable improvements around DateTime

Improvements

• Consistent FrozenTime usage for datetime handling to prevent timezone issues (DB vs PHP) for delayed jobs.
• Improved updateProgress() to allow status update.
• Improve pending widget overview (include delay, progress and status)
• Allow delayed job execution to be tested from the admin backend.
• Include detailed stats per job type. Needs to be enabled via isStatisticEnabled config.

Improvements

• Better isQueued() handling with optional job type
• CakePHP 3.6+, deprecations removed
• Phinx 0.10 compatibility

Bugfixes

• Fixed exclude lists for types or groups.
• Use date() around timestamp usage.

Bugfixes

Make sure that if a process gets stuck, e.g. in some exec() forever running code, that the crontab doesnt built up too many never-ending workers running and then killing the server.

Default: 120s*100 = 3.3h

So if you spawn new runners every 5 minutes via crontab cronjob setting, you get a total process count of 40 as the worse case scenario.

Via workertimeout config it can be manually adjusted to a custom value.
Make sure you adjust it to the longest possible running job * x seconds at least.

The previous 0 (forever) timeout on runners is not a recommended setting anymore.

Improvements

Allow filtering per type and group also with multiple values (comma separated). Allow also excluding using - prefix.

Improvements

• Allow runner to only fetch certain group or type.
• Search functionality can be disabled via Queue.isSearchEnabled Configure key. Otherwise auto-detect based on friendsofcake/search plugin being loaded decides this.

Improvements

• Added Monitor example task that can also be used to quickly check the current PHP and memory setting in the (crontab) CLI environment used for the cronjob runner.
• Added filtering for QueuedJobs listing, requires friendsofcake/search plugin to be installed and loaded.
• Display priority in queue backend overview and a quick link to the job details.
• Allow memory limit info to be logged with each job.
• Removed more deprecations.