bullmq

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis

MIT License

Downloads
3.3M
Stars
6K
Committers
152

๐Ÿ›  Tutorials

You can find tutorials and news in this blog: https://blog.taskforce.sh/

News ๐Ÿš€

๐ŸŒ Language agnostic BullMQ

Do you need to work with BullMQ on platforms other than Node.js? If so, check out the BullMQ Proxy

๐ŸŒŸ Rediscover Scale Conference 2024

Discover the latest in in-memory and real-time data technologies at Rediscover Scale 2024. Ideal for engineers, architects, and technical leaders looking to push technological boundaries. Connect with experts and advance your skills at The Foundry SF, San Francisco.

Learn more and register here!

Official FrontEnd

Supercharge your queues with a professional front end:

  • Get a complete overview of all your queues.
  • Inspect jobs, search, retry, or promote delayed jobs.
  • Metrics and statistics.
  • and many more features.

Sign up at Taskforce.sh

๐Ÿš€ Sponsors ๐Ÿš€

Used by

Some notable organizations using BullMQ:

The gist

Install:

$ yarn add bullmq

Add jobs to the queue:

import { Queue } from 'bullmq';

const queue = new Queue('Paint');

queue.add('cars', { color: 'blue' });

Process the jobs in your workers:

import { Worker } from 'bullmq';

const worker = new Worker('Paint', async job => {
  if (job.name === 'cars') {
    await paintCar(job.data.color);
  }
});

Listen to jobs for completion:

import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents('Paint');

queueEvents.on('completed', ({ jobId }) => {
  console.log('done painting');
});

queueEvents.on(
  'failed',
  ({ jobId, failedReason }: { jobId: string; failedReason: string }) => {
    console.error('error painting', failedReason);
  },
);

This is just scratching the surface, check all the features and more in the official documentation

Feature Comparison

Since there are a few job queue solutions, here is a table comparing them:

Feature BullMQ-Pro BullMQ Bull Kue Bee Agenda
Backend redis redis redis redis redis mongo
Observables โœ“
Group Rate Limit โœ“
Group Support โœ“
Batches Support โœ“
Parent/Child Dependencies โœ“ โœ“
Debouncing โœ“ โœ“ โœ“
Priorities โœ“ โœ“ โœ“ โœ“ โœ“
Concurrency โœ“ โœ“ โœ“ โœ“ โœ“ โœ“
Delayed jobs โœ“ โœ“ โœ“ โœ“ โœ“
Global events โœ“ โœ“ โœ“ โœ“
Rate Limiter โœ“ โœ“ โœ“
Pause/Resume โœ“ โœ“ โœ“ โœ“
Sandboxed worker โœ“ โœ“ โœ“
Repeatable jobs โœ“ โœ“ โœ“ โœ“
Atomic ops โœ“ โœ“ โœ“ โœ“
Persistence โœ“ โœ“ โœ“ โœ“ โœ“ โœ“
UI โœ“ โœ“ โœ“ โœ“ โœ“
Optimized for Jobs / Messages Jobs / Messages Jobs / Messages Jobs Messages Jobs

Contributing

Fork the repo, make some changes, submit a pull-request! Here is the contributing doc that has more details.

Thanks

Thanks for all the contributors that made this library possible, also a special mention to Leon van Kammen that kindly donated his npm bullmq repo.