How to Use Bull and Redis to Build Fast, Reliable Background Jobs in Node.js

js

How to Use Bull and Redis to Build Fast, Reliable Background Jobs in Node.js

Learn how to improve app performance and user experience by offloading tasks with Bull queues and Redis in Node.js.

Dec 18, 2025

How to Use Bull and Redis to Build Fast, Reliable Background Jobs in Node.js

I’ve been thinking about how we handle tasks in our applications. You know those moments when a user clicks a button and then waits… and waits? Maybe they’re uploading a photo, or requesting a report, or sending an invitation to fifty people. The application freezes, the spinner spins, and everyone gets frustrated. There has to be a better way.

What if we could say, “Got it, we’ll handle that for you,” and let the user move on immediately? That’s the power of moving work to the background. It transforms the user experience from waiting to doing. This approach isn’t just about speed; it’s about building applications that feel responsive and reliable, even under heavy load.

Let’s talk about how to make this happen. We’ll use a tool called Bull, which is a queue system built on Redis. Think of it as a super-organized to-do list for your application. Instead of doing everything right away, you write tasks down on this list, and dedicated workers pick them up and complete them, one by one or many at a time.

Why does this matter? Well, have you ever had an email fail to send and crash your entire registration process? With a queue, that email job can fail, retry on its own, and your user still gets their account. The core function is protected.

First, we need to set up our project. You’ll need Node.js and a Redis server running. Redis is just a very fast data store that Bull uses to keep track of everything. Let’s install what we need.

npm install bull redis

Now, let’s create a simple queue. We’ll make one for sending emails, a classic background job.

// emailQueue.js
const Queue = require('bull');
const redisConfig = { host: 'localhost', port: 6379 };

const emailQueue = new Queue('email sending', { redis: redisConfig });

module.exports = emailQueue;

We’ve created a queue named ‘email sending’. It’s connected to our local Redis. This queue object is our gateway. We can add jobs to it, and we can set up workers to process those jobs. They are separate concerns, which is key.

Adding a job is straightforward. Imagine a user signs up, and we need to send a welcome email.

// In your user signup route
const emailQueue = require('./emailQueue');

app.post('/signup', async (req, res) => {
  // 1. Create the user in your database
  const newUser = await createUser(req.body);

  // 2. Add an email job to the queue and respond immediately
  await emailQueue.add({
    to: newUser.email,
    subject: 'Welcome!',
    template: 'welcome',
    userId: newUser.id
  });

  // 3. Send success response right away
  res.json({ success: true, message: 'Account created! Check your email.' });
});

See what happened? The user gets an instant response. The email is now someone else’s problem—the queue worker’s. The main application thread is free to handle the next request. This is how you build a snappy API.

But who processes the job? We need a worker. This can be in the same codebase or a completely separate service.

// emailWorker.js
const Queue = require('bull');
const redisConfig = { host: 'localhost', port: 6379 };
const emailQueue = new Queue('email sending', { redis: redisConfig });
const sendEmail = require('./emailService'); // Your real email logic

emailQueue.process(async (job) => {
  console.log(`Processing job ${job.id}: Sending to ${job.data.to}`);
  
  // This is where the actual work happens
  await sendEmail(job.data);
  
  // Return a result if needed
  return { status: 'sent', to: job.data.to };
});

This worker file runs continuously. It listens to the ‘email sending’ queue. When a new job appears, it takes it, runs our sendEmail function, and marks it as done. If sendEmail throws an error, Bull can automatically retry the job. How many times would you want it to retry before giving up?

This separation is powerful. You can scale your workers independently. If you’re getting a flood of signups, you can launch more instances of emailWorker.js to process emails faster, without touching your main web server.

Jobs can be more than just “do this now.” They can be scheduled. Need to send a reminder email in 24 hours?

// Schedule a job for one day from now
const delay = 24 * 60 * 60 * 1000; // 24 hours in milliseconds
await emailQueue.add(
  { to: userEmail, subject: 'Your Reminder' },
  { delay: delay }
);

The job will sit in the queue, waiting, and will only become available for workers after that delay has passed. It’s a built-in, persistent scheduler.

What about monitoring? You can’t manage what you can’t see. Bull provides events for everything.

emailQueue.on('completed', (job, result) => {
  console.log(`Job ${job.id} finished. Result:`, result);
});

emailQueue.on('failed', (job, err) => {
  console.error(`Job ${job.id} failed with error:`, err.message);
  // Maybe alert a developer here
});

emailQueue.on('stalled', (job) => {
  console.warn(`Job ${job.id} stalled. It might be retried.`);
});

These events let you keep a pulse on your system. Is there a spike in failures? Maybe your email service provider is down. Are jobs stalling? Maybe your workers are overloaded.

Let’s look at a more complex example: processing an uploaded image. This is CPU-intensive and perfect for a background job.

// imageQueue.js
const imageQueue = new Queue('image processing', { redis: redisConfig });

// In your upload route
app.post('/upload', upload.single('photo'), async (req, res) => {
  const job = await imageQueue.add({
    userId: req.user.id,
    filePath: req.file.path,
    sizes: ['thumb', 'medium', 'large']
  });
  
  res.json({ jobId: job.id, status: 'processing' });
});

// The worker
imageQueue.process(5, async (job) => { // Process 5 jobs concurrently
  const { filePath, sizes } = job.data;
  
  for (const size of sizes) {
    await createImageSize(filePath, size); // Your resizing logic
    job.progress((sizes.indexOf(size) + 1) / sizes.length * 100);
  }
  
  return { originalPath: filePath, processedSizes: sizes };
});

Notice the 5 in imageQueue.process(5, ...). This tells Bull this worker can process up to 5 jobs from this queue at the same time. For I/O or network-bound tasks (like emails), you can set this number quite high. For CPU-bound tasks (like image processing), you’ll want to match it to your server’s cores.

Also, see job.progress(). This is a fantastic feature. You can report progress from within a long-running job. Then, on your frontend, you could poll an API that checks the job’s status and progress, showing the user a progress bar for their upload. It turns a black box into a transparent process.

But what goes wrong? Plenty. Network timeouts, third-party API limits, memory issues. Bull handles retries gracefully. You can configure them when adding a job.

await emailQueue.add(data, {
  attempts: 5, // Try up to 5 times
  backoff: {
    type: 'exponential', // Wait 2s, then 4s, then 8s...
    delay: 2000
  }
});

Exponential backoff is a good neighbor policy. If a service is down, hammering it every second makes things worse. Waiting longer between each attempt is more polite and often more successful.

Sometimes, a job fails all its attempts. You don’t want to lose it. This is where a “dead letter queue” concept comes in. You can listen for failed jobs and move them to a separate queue for manual inspection.

const failedEmailQueue = new Queue('failed-emails');

emailQueue.on('failed', async (job, err) => {
  // After all retries, move to a different queue
  if (job.attemptsMade >= job.opts.attempts) {
    await failedEmailQueue.add(job.data, {
      ...job.opts,
      failedReason: err.message
    });
    console.log(`Moved job ${job.id} to failed queue.`);
  }
});

This way, no data is silently lost. An admin can later check the failed-emails queue, see the error, fix the underlying issue (like updating an API key), and retry the jobs.

Getting this right changes how you design systems. You start thinking in terms of “commands” and “events.” The user action (sign up) issues a command (send welcome email). The queue system ensures it happens. If you later need to add a second action (like adding the user to a newsletter), you just add another job to the queue. The signup route doesn’t change. It’s incredibly flexible.

The mental shift is from synchronous, linear execution to asynchronous, event-based flow. Your application becomes a coordinator of work, not the sole doer of work. This makes it more robust. A worker can crash, and when it restarts, it will pick up where it left off because Redis persists the queue state.

Start small. Take one slow operation in your app—sending a notification, generating a PDF, cleaning up old files—and move it to a Bull queue. You’ll immediately feel the improvement in responsiveness. Then, you’ll start seeing opportunities everywhere. The question becomes not “can this be queued?” but “why is this not queued?”

I hope this gives you a clear path to making your applications more responsive and reliable. This pattern has saved me countless times from midnight alerts about timeouts and crashes. It’s a foundational piece of modern application design.

If you found this walkthrough helpful, please share it with a colleague who might be battling with slow requests. Have you tried implementing a queue before? What was your biggest challenge? Let me know in the comments—I’d love to hear about your experiences and answer any questions.

As a best-selling author, I invite you to explore my books on Amazon. Don’t forget to follow me on Medium and show your support. Thank you! Your support means the world!

101 Books

101 Books is an AI-driven publishing company co-founded by author Aarav Joshi. By leveraging advanced AI technology, we keep our publishing costs incredibly low—some books are priced as low as $4—making quality knowledge accessible to everyone.

Check out our book Golang Clean Code available on Amazon.

Stay tuned for updates and exciting news. When shopping for books, search for Aarav Joshi to find more of our titles. Use the provided link to enjoy special discounts!

📘 Checkout my latest ebook for free on my channel!
Be sure to like, share, comment, and subscribe to the channel!

Our Creations

Be sure to check out our creations:

We are on Medium

Share: Facebook Twitter Reddit LinkedIn WhatsApp Telegram Pinterest Email Instagram

js