What are Queues in Laravel?

What are Queues?

Queues are a powerful mechanism in Laravel that allow you to defer time-consuming tasks, such as sending emails or generating reports, to be processed in the background. Instead of executing tasks like these immediately, Laravel allows you to place them in a queue, where they are processed later by a background worker.

Why use Queues?

Queues offer several benefits as:

• Improved User Experience: By processing tasks in the background, users don’t have to wait for operations to complete.
• Efficiency: They optimize system resources by distributing tasks across workers, handling multiple tasks at once.
• Scalability: Easily handle large loads by distributing tasks across multiple workers or servers.

Setting Up Queues

Before starting to use queues, you'll have to set them up in Laravel.

Configure the Queue Driver

Laravel supports various queue drivers like database, Redis, Amazon SQS, and more. By default, the sync driver is used, which processes jobs synchronously, which means no queue is configured. To change this, modify the QUEUE_CONNECTION value in the .env file:

QUEUE_CONNECTION=database

Then you'll have to migrate the database table for storing jobs:

php artisan queue:table
php artisan migrate

Running the Queue Worker

Once you’ve set up the queue, you can start the queue worker to process jobs in the background. The following command starts a worker that listens for new jobs and processes them:

php artisan queue:work

Alternatively, you can use queue:listen, which restarts the worker after each job, making it easier for development but less efficient for production.

Creating a Job

To use queues, you need to create jobs that define the tasks to be processed in the background. Laravel provides a simple Artisan command to generate job classes.

Generating a Job

The following command creates a new Job inside the app/jobs/ directory:

php artisan make:job SendWelcomeEmail

Now, you can define the job’s behavior in the handle() method.

Dispatching Jobs to Queues

After creating a job, you can dispatch it to a queue. There are various way to dispatch jobs in Laravel, including delaying jobs for later execution.

To dispatch a job, you can use the dispatch() method. You can place this wherever the action should be triggered, like in a controller or service.

SendWelcomeEmail::dipatch($email);

Delayed Jobs

If you want the job to run after a certain time has passed, you can delay it like this:

SendWelcomeEmail::dipatch($email)->delay(now()->addMinutes(5));

This delays the execution of the job for 5 minutes.

Handling Failed Jobs

Once your application is using queues, it's essential to monitor and manage your queue workers, because inevitably some jobs might fail due to issues like network errors or unhandled exceptions.

Laravel provides built-in mechanisms to log, inspect and retry failed jobs, ensuring that you can address issues promptly. You can configure the failed job driver in your application to log failures in a database table and use Artisan commands to manage these jobs.

For more information about failed jobs and retrying them, check Laravel's official documentation.

Practical Example: Newsletter Subscriptions

Now I'll walk you through a practical example where you use queues to send a welcome email after a user subscribes to your newsletter. Instead of sending the email synchronously, which could slow down the subscription process, I'll dispatch the task to a queue, allowing it to be processed in the background.

Step 1: Create a Job for Sending the Welcome Email

First, you'll have to create a new job for sending the welcome email:

php artisan make:job SendWelcomeEmail

Open the generated file in app/Jobs/SendWelcomeEmail.php and implement the email-sending logic:

namespace App\Jobs;

use App\Mail\WelcomeNewsletter;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Mail;

class SendWelcomeEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $email;

    public function __construct($email)
    {
        $this->email = $email;
    }

    public function handle()
    {
        // Send the welcome email
        Mail::to($this->email)->send(new WelcomeNewsletter());
    }
}

In the code above, the SendWelcomeEmail job sends a WelcomeNewsletter email to the user. The job implements a ShouldQueue interface indicating that it will be queued.

Step 2: Create the Newsletter Controller

In the controller, you dispatch the SendWelcomeEmail job when the user subscribes to the newsletter.

namespace App\Http\Controllers;

use App\Jobs\SendWelcomeEmail;
use Illuminate\Http\Request;

class NewsletterController extends Controller
{
    public function subscribe(Request $request)
    {
        // Validate email input
        $validated = $request->validate([
            'email' => 'required|email'
        ]);

        // Dispatch job to send welcome email
        SendWelcomeEmail::dispatch($validated['email']);

        return response()->json(['message' => 'Subscription successful! A welcome email will be sent shortly.']);
    }
}

Once the user submits their email, the subscribe method validates it and then dispatches the SendWelcomeEmail job to the queue.

Step 3: Running the Queue Worker

Once everything is set up, you can start the queue worker to begin processing the jobs:

php artisan queue:work

As soon as the worker runs, it will process the SendWelcomeEmail job in the background, and the user will receive their welcome email without any delay in the subscription process.

Step 4: Handling Failed Jobs

To ensure the newsletter subscription process is robust, you should handle potential failures when sending the welcome email. Laravel allows you to log and manage failed jobs, so you can retry or investigate them later.

Configuring Failed Job Logging

First, ensure that failed job logging is enabled by setting the QUEUE_FAILED_DRIVER in the .env file:

QUEUE_FAILED_DRIVER=database

Run the migration to create the failed_jobs table where your failed jobs will be stored:

php artisan queue:failed-table
php artisan migrate

Customizing Job Failure Handling

You can customize the job to take specific actions when it fails. In the SendWelcomeEmail job, you can use the failed() method to define what should happen if the job fails. For example, you might want to log the error or notify an administrator:

use Illuminate\Support\Facades\Log;

class SendWelcomeEmail implements ShouldQueue
{
    // ... existing code ...
    
    public function failed(\Exception $exception)
    {
        // Log the error or notify an administrator
        Log::error('Failed to send welcome email to ' . $this->email . ': ' . $exception->getMessage());
    }
}

By setting up failed job handling, you ensure that no email is missed, and you have the ability to debug issues that arise during the queue processing.

And that's it, now your newsletter can send welcome mails to new subscribers!

Best Practices

• Keep Jobs Small: Jobs should be quick to execute. Break complex processes into multiple jobs if necessary.

• Retry Logic: Use Laravel’s retry mechanisms to automatically retry failed jobs.

• Handle Failed Jobs: Set up failed job logging to track and manage jobs that fail to execute.

• Use Delays Wisely: If your jobs rely on external services, such as sending emails, consider adding delays between retries to avoid hitting API rate limits.

Useful Links

• 📑 Laravel Documentation: Queues