Async Processing / Job Queues

Also Known As

TL;DR

Explanation

Synchronous processing of slow operations (sending emails, resizing images, calling external APIs) directly in a web request increases response times and risks timeouts. Job queues (Laravel Queues, Symfony Messenger, RabbitMQ, Redis queues) accept a job description, store it durably, and process it asynchronously in a separate worker process. This reduces web response time to near-instant, improves resilience (failed jobs can be retried), and allows work to be distributed across multiple workers. The trade-off is eventual consistency and added infrastructure complexity.

Diagram

flowchart LR
    subgraph Synchronous - Slow
        REQ1[HTTP Request] --> PROC1[Process<br/>resize image<br/>5 seconds] --> RES1[Response<br/>after 5s]
    end
    subgraph Async - Fast
        REQ2[HTTP Request] --> QUEUE[Push to Queue<br/>immediately] --> RES2[Response<br/>50ms]
        QUEUE --> WORKER[Background Worker<br/>resizes image]
        WORKER --> NOTIFY[Notify user<br/>when done]
    end
    style RES1 fill:#f85149,color:#fff
    style RES2 fill:#238636,color:#fff

Common Misconception

Why It Matters

Common Mistakes

• Dispatching jobs to a queue but not running any queue workers — jobs sit queued and never process.
• Putting too much serialised data in the job payload — pass IDs and re-fetch fresh data in the worker instead.
• Not handling job failures with retries and dead-letter queues — failed jobs silently disappear.
• Running queue workers without a process supervisor (Supervisor, Horizon) — workers die and no one notices.

Code Examples

// Synchronous — user waits for email + PDF + analytics:
function placeOrder(Order $o): void {
    $this->db->save($o);
    $this->mailer->sendConfirmation($o);  // 800ms
    $this->pdf->generateInvoice($o);      // 1200ms
    $this->analytics->track($o);          // 300ms
    // User blocked for 2.3s — all should be queued
}

// Dispatch heavy work to a queue — return immediately to the user
class OrderController {
    public function store(Request $req): JsonResponse {
        $order = Order::create($req->validated());

        // Don't process payment synchronously — queue it
        ProcessPayment::dispatch($order)->onQueue('payments');
        GenerateInvoice::dispatch($order)->onQueue('documents');
        SendConfirmationEmail::dispatch($order)->delay(now()->addSeconds(5));

        return response()->json(['id' => $order->id], 202); // Accepted
    }
}

// Worker processes jobs in the background
$ php artisan queue:work --queue=payments,documents,default

References

• ↗ https://laravel.com/docs/queues
• ↗ https://symfony.com/doc/current/messenger.html

Tags

Related Terms