Building a Queue System in Node.js with BullMQ

Introduction

Background job processing is essential for offloaded tasks. This guide shows how to build a reliable queue system in Node.js using BullMQ and Redis.

Prerequisites

• Node.js >=16
• Redis instance

Step 1: Initialize Project

mkdir node-queue-bullmq
cd node-queue-bullmq
npm init -y
npm install bullmq ioredis

Step 2: Create Queue and Worker

Create queue.js:

const { Queue } = require("bullmq");
const Redis = require("ioredis");

const connection = new Redis();
const myQueue = new Queue("myQueue", { connection });

async function addJob(data) {
  await myQueue.add("myJob", data);
}

module.exports = { addJob };

Create worker.js:

const { Worker, QueueEvents } = require("bullmq");
const Redis = require("ioredis");

const connection = new Redis();
const queueName = "myQueue";

new QueueEvents(queueName, { connection });

const worker = new Worker(
  queueName,
  async job => {
    // Process job
    console.log(`Processing job ${job.id} with data`, job.data);
  },
  { connection }
);

worker.on("completed", job => {
  console.log(`Job ${job.id} completed`);
});

Step 3: Use the Queue

Create index.js:

const { addJob } = require("./queue");

(async () => {
  await addJob({ task: "sendEmail", to: "user@example.com" });
})();

Step 4: Run Services

node worker.js  # start worker
node index.js   # add a job

Summary

BullMQ provides a powerful, Redis-backed queue system with events and job processing for scalable background tasks.