When should you use a queue?

Apply one test: can this be done later? If the answer is yes, a Queue almost always makes the system better. When a user uploads a video, they need to know the upload succeeded, not that every format is encoded this second, so encoding can be done later. When a payment goes through, the receipt email can arrive 30 seconds afterward. Both belong on a Queue. But when a user requests their bank balance, they are staring at the screen waiting for a number, so that is a Service call, not a Queue job. The "done later" test is how you decide. Match the block to whether the user is actually waiting.

How do queues make systems more resilient?

Three properties. First, spike absorption: when a million messages pile up during a viral moment, the Queue holds them and your Workers chew through them at a steady pace, so the system gets temporarily behind instead of going down. There is a huge difference between slow and down. Second, failure isolation: if the encoding Worker crashes, the upload Service does not care, it just keeps dropping messages on the Queue, and the Worker picks up where it left off when it restarts. No data lost, no user-facing errors. Third, independent scaling: more uploads, add Service instances; encoding falling behind, add Workers. The Queue mediates so each part scales on its own needs.

When should you not use a queue?

When the user is actively waiting for a result that determines their next action. If a request fails the "done later" test, a direct Service call is the right choice and a Queue just adds latency with no benefit. Authentication is the clear case: users cannot wait for their login to "process later." So are search queries (the user is typing and expects results now), balance checks (financial data must be current), and real-time form validation (it needs instant feedback). The common thread is that the user cannot move forward until the answer comes back. A Queue is for work that can wait. When nothing can wait, do not reach for one.

← Back to Learning Hub 6 min read

7 Building Blocks Framework

Queues: The Building Block That Makes Systems Resilient

A queue forces one uncomfortable question: can this be done later? That question is the difference between fragile and resilient.

The Building Block AI Always Forgets

Of all the 7 building blocks, the queue is the one AI coding assistants almost never suggest. Ask an AI to build a video upload feature and it will generate a service that handles the upload, processes the video, and returns a response. Clean, direct, synchronous. And fragile.

The queue is the building block that makes systems survive the real world. Not the world of demos and happy paths, but the world where a million users upload videos at the same time, where downstream services go down at 2 AM, and where a traffic spike from a viral post could take your entire platform offline.

What a Queue Actually Does

Queue Block

A queue holds messages. That is it. One component puts a message in. Another component takes a message out. The two components never talk to each other directly.

This sounds almost too simple to matter. But that separation is what makes systems resilient.

# Producer: the upload service
def handle_upload(request):
    video_id = save_raw_video(request.file)
    queue.send({
        "video_id": video_id,
        "formats": ["1080p", "720p", "480p"],
        "uploaded_by": request.user_id
    })
    return {"status": "processing", "video_id": video_id}

The upload service does not encode the video. It does not even know how encoding works. It saves the raw file, drops a message on the queue, and tells the user "we are processing your video." Done in under a second.

On the other side, a worker pulls messages from the queue and does the slow, heavy encoding work.

Worker Block If it takes 5 minutes per video, that is fine. The user is not waiting. The queue holds the work until the worker is ready.

The "Done Later" Insight

The most important architectural question a queue forces you to ask is: can this be done later?

That question sounds simple. It is not. It requires you to think about user expectations, not just technical implementation.

When a user uploads a video, do they need the encoded versions immediately? No. They need to know the upload succeeded. They can wait a few minutes for processing. Done later is acceptable.

When a user submits a payment, do they need the receipt email immediately? No. They need confirmation that the payment went through. The email can arrive 30 seconds later. Done later is acceptable.

When a user requests their bank balance, can that be done later? No. They are staring at the screen waiting for a number. That is a service call, not a queue job.

The "done later" test is how you decide whether a queue belongs in your design. If the answer is yes, a queue almost always makes the system better. If the answer is no, a queue adds unnecessary complexity.

Why Queues Make Systems Resilient

Three properties make queues the backbone of resilient architecture:

1. Spike Absorption

Without a queue, your system can only handle as many requests as your workers can process simultaneously. If you have 10 workers and each takes 1 second, you handle 10 requests per second. The 11th request either waits (slow) or fails (broken).

With a queue, spikes get absorbed. A million messages can pile up during a viral moment, and your workers chew through them at a steady pace. The system never breaks. It just gets temporarily behind. There is a huge difference between "slow" and "down."

2. Failure Isolation

Without a queue, if the video encoding service crashes, the upload service has nowhere to send work. Uploads fail. Users see errors.

With a queue, the upload service does not care whether the encoding worker is running. It drops messages on the queue and moves on. If the worker crashes, messages accumulate on the queue. When the worker restarts, it picks up where it left off. No data lost. No user-facing errors.

This is failure isolation. The queue acts as a buffer between components, preventing failures in one part of the system from cascading to others.

3. Independent Scaling

Without a queue, your upload service and your encoding logic are tangled together. To handle more uploads, you scale both, even if uploads are fast and encoding is the bottleneck.

With a queue, you scale them independently. Getting more uploads? Add upload service instances. Encoding falling behind? Add more workers. The queue mediates between them, so each component scales based on its own needs.

The Tradeoff: Managing Expectations

Queues are not free. They introduce a user experience challenge: you have to tell users "this will be done later."

That means designing your interface for eventual completion. A progress bar. A notification when processing finishes. A "your video is being processed" state that does not look like an error.

This is a product decision, not just a technical one. The queue makes the system resilient, but only if the product design accounts for the delay.

Instagram handles this beautifully. When you post a photo, the app shows it immediately in your feed. Behind the scenes, workers are generating thumbnails, running content moderation, and updating follower feeds. You see instant feedback. The heavy work happens later.

When Not to Use a Queue

Not everything belongs in a queue. If you fail the "done later" test, a direct service call is the right choice.

Authentication: Users cannot wait for their login to "process later"
Search queries: The user is typing and expects results now
Balance checks: Financial data must be current
Real-time validation: Form validation needs instant feedback

The common thread: these are all cases where the user is actively waiting for a result that determines their next action. A queue would add latency with no benefit.

Why AI Defaults to Direct Calls

AI coding assistants generate the simplest path from input to output: request comes in, work happens, response goes out. A direct call works correctly. It just does not work reliably under load, under failure, or under the unpredictable conditions of production.

Knowing when to introduce a queue requires thinking about what happens when things go wrong. AI does not think about those scenarios unless you explicitly tell it to. And knowing what to tell it is your value as an engineer.

The Queue in Context

The queue connects the other building blocks. It sits between services and workers, turning synchronous requests into asynchronous work. It works alongside the storage blocks that hold the data being processed.

In most production systems, the pattern looks like this: a service receives a request, stores the raw data, places a message on a queue, and returns immediately. A worker picks up the message, does the heavy processing, and stores the result. The user checks back (or gets notified) when the work is done.

That pattern powers Instagram uploads, Netflix video encoding, Uber ride matching, and Slack message delivery. Different products, same building blocks, same architecture.

Test Your Understanding

Ready to decide where queues belong in a real system? The interactive challenges put you in the architect's seat. You will design Instagram, Netflix, and Uber, and the queue decisions are some of the most revealing choices you will make.

And if you want to go deeper into the complete framework, start from the beginning: The 7 Building Blocks Behind Every System.

Frequently Asked Questions

What is a message queue?: A Queue holds messages. That is the whole job. One component puts a message in, another component takes a message out, and the two never talk to each other directly. An upload Service drops a "process this video" message on the Queue and immediately tells the user "we are processing your video," done in under a second. On the other side, a Worker pulls messages off and does the slow, heavy work, like encoding the video, which might take five minutes. The user is not waiting on that. The Queue holds the work until the Worker is ready. That simple separation is what makes a system resilient instead of fragile.
When should you use a queue?: Apply one test: can this be done later? If the answer is yes, a Queue almost always makes the system better. When a user uploads a video, they need to know the upload succeeded, not that every format is encoded this second, so encoding can be done later. When a payment goes through, the receipt email can arrive 30 seconds afterward. Both belong on a Queue. But when a user requests their bank balance, they are staring at the screen waiting for a number, so that is a Service call, not a Queue job. The "done later" test is how you decide. Match the block to whether the user is actually waiting.
How do queues make systems more resilient?: Three properties. First, spike absorption: when a million messages pile up during a viral moment, the Queue holds them and your Workers chew through them at a steady pace, so the system gets temporarily behind instead of going down. There is a huge difference between slow and down. Second, failure isolation: if the encoding Worker crashes, the upload Service does not care, it just keeps dropping messages on the Queue, and the Worker picks up where it left off when it restarts. No data lost, no user-facing errors. Third, independent scaling: more uploads, add Service instances; encoding falling behind, add Workers. The Queue mediates so each part scales on its own needs.
When should you not use a queue?: When the user is actively waiting for a result that determines their next action. If a request fails the "done later" test, a direct Service call is the right choice and a Queue just adds latency with no benefit. Authentication is the clear case: users cannot wait for their login to "process later." So are search queries (the user is typing and expects results now), balance checks (financial data must be current), and real-time form validation (it needs instant feedback). The common thread is that the user cannot move forward until the answer comes back. A Queue is for work that can wait. When nothing can wait, do not reach for one.

Don't just read it — build with it

You can't read your way to judgment. Put this block into practice: map your own app with Design with Blocks, or play the building blocks game with Instagram, Netflix, and Uber.

Design with Blocks Play the Game