The power of Laravel Queues

Last updated: July 29th 2021

laravel queues Become an author

Introduction

While building your web application, you may have some tasks, that require more computer power or time than a typical web request can provide if you make the task run synchronously. If the user have to wait for a long time for the application to complete the task, it makes up for at bad user experience. The answer to this is to make the task asynchronous and run in the background while the user can procced their interaction with your application.

Laravel queues provide an API to use several queue backends. We will cover two options, eg. how to set up Redis and a relational database, that both comes out of the box on our Perfect Server Stacks. You can choose to use one or both of them in your application.

In this article we will build a simple queue system for our fictional freemium SAAS application, where we will send the new user a notification/email 5 days after registration, trying to onboard them to the paid service.

Prerequisites

Spin up one of our perfect servers. You will also need an understanding of Laravel observers and emails.

Key concepts

  • Jobs - tasks that stack on to one or more queues.
  • Workers - processes the job. If no worker is being active, no jobs will be proccesed. An important thing during development is to remember to restart your worker everytime you change a job. If not done, your changes won't take effect. Do it by typing php artisan queue:restart into the console or you can type php artisan queue:listen as an alternative to queue:work that lets you watch the queue process so you don’t have to manually restart the worker after code changes.
  • Serialization - detects the eloquent model and serializes the models attributes for the job to use. Also deserializes the model for the worker to handle the jobs logic. Happens automatically in the Laravel pipeline.
  • Daemons - means that the worker you set up to execute your tasks is stored in memory, thus not reflecting changes you make to your code before restarted. Ultimatly you will need to restart the worker every time you deploy new code to your production server.
  • Delayed job execution - delay() helper function. Specific to the functionallity of this article example.

Application preparation

Follow along to get some boilerplate code to work with.

First, install the Laravel Breeze package and modify the users migration to determine if they are subscribed or not. To do so add a subscribed boolean as follows - file path: database/migrations/create_users_table. Addition marked in bold

		public function up()
	    {
	        Schema::create('users', function (Blueprint $table) {
	            $table->id();
	            $table->string('name');
	            $table->string('email')->unique();
	            $table->timestamp('email_verified_at')->nullable();
	            $table->string('password');
	            $table->boolean('subscribed')->default(false);
	            $table->rememberToken();
	            $table->timestamps();
	        });
	    }

Migrate the migration files

	    php artisan migrate

Next up, generate a mail class in the console

		php artisan make:mail UserOnboarding

Build the mail - file path: app/Mail/UserOnboarding.

	    public function build()
	    {
	        return $this->from('example@webdock.io')
	        ->subject('Queued Email')
	        ->view('emails.user.onboarding');
	    }

Then create a new blade template to hold the mail view, we called it onboarding.blade.php and placed it in an user/emails subfolder. Modify the email template to your liking - file path: resources/views/emails/user/onboarding.blade.php

	    <!DOCTYPE html>
		<html>
		<head>
			<title>Webdock Laravel Queue</title>
		</head>
		<body>
			<p>Test email.</p>
		</body>
		</html>

Create and set up the job class for the registration onboarding process. The console command will generate a new folder in the app directory. In the console, type:

		php artisan make:job UserOnboardingEmail

Typehint the user in the constructor to leverage Laravels route-model binding and in the handle method you will specify the user and the email class - file path: app/Jobs/UserOnboardingEmail.

		class UserOnboardingEmail implements ShouldQueue
		{
			protected $user;

			public function __construct(User $user)
			{
				$this->user = $user;
			}

			public function handle()
		    {
		    	// Check if the user is paying
		    	if (!$this->user->subscribed) {
	                Mail::to($this->user->email)->send(new UserOnboarding($this->user));
		    	}
		    }
		}

Set up an event listener if this job is the only thing to listen for in the user model. If more actions are required, use an observer class if you later want to listen for many events on a given model to group all of your listeners into a single class. We will do the latter. This console command will also generate a new subfolder in the app directory for you.

		php artisan make:observer UserObserver --model=User

Instantiate the UserOnboardingEmail job in the UserObserver class. Notice the delay() function that will ensure the email is sent 5 days after registration - file path: app/Observers/UserObserver

		class UserObserver
		{
		    public function created(User $user)
		    {
		        UserOnboardingEmail::dispatch($user)->delay(now()->addDays(5));
		    }
		}

And registrer in the EventServiceProvider - file path: app/Providers/EventServiceProvider.

		public function boot()
	    {
	        User::observe(UserObserver::class);
	    }

Failed jobs

Sometimes jobs will fail. After a job fails it will be inserted into the failed_jobs database table which comes out off the box with your Laravel installation.

To retry failed jobs, add the all flag to the queue:retry command

		php artisan queue:retry all

To delete all of your failed jobs from the failed_jobs table, use the queue:flush command

		php artisan queue:flush

Excecuting jobs using Redis

Setting up Redis

On our perfect stacks, Redis is already installed so setting it up is simple. First install the redis composer package via the console and then locate the .env file and change the QUEUE_CONNECTION to redis.

Console

		composer require predis/predis

.env file

		QUEUE_CONNECTION=redis

Running php artisan queue:work in the console should now be enough.

Excecuting jobs using a relational database

In order to use the database queue driver, you will need a database table to hold the jobs. To generate a migration that creates this table, run the queue:table Artisan command. Once the migration has been created, you may migrate your database using the migrate command. Finish by changing the QUEUE_CONNECTION in your environment file.

Console

		php artisan queue:table
		php artisan migrate

.env

		# .env
		QUEUE_CONNECTION=database

Running php artisan queue:work should do the trick.

Keep the queue worker running on a production server

In production, a queue:work process may stop running if you close the console, exceeds worker timeout or restart the worker. To handle this situation you can configure a process monitor that watch your worker(s) and restart them if they exit for some of the before mentioned reasons. Supervisor is our favorite tool and the one recommended by Laravel itself. Get it by typing the following in the console:

		sudo apt-get install supervisor

Supervisor configuration files are typically stored in the /etc/supervisor/conf.d directory. Create a configuration file for Supervisor to monitor the workers. We call it worker.conf:

		cd /etc/supervisor/conf.d
		touch worker.conf

And fill it as follows:

		[program:worker]
		process_name=%(program_name)s_%(process_num)02d
		command=php /path/to/your/laravel/installation/artisan queue:work
		autostart=true
		autorestart=true
		stopasgroup=true
		killasgroup=true
		user=youruser
		numprocs=1
		redirect_stderr=true
		stdout_logfile=/path/to/your/laravel/installation/worker.log
		stopwaitsecs=3600

For more information about Supervisor and it's functionallity, consult the Supervisor documentation.

Once the configuration file has been created, update the Supervisor configuration and start the processes, again from the console:

		sudo supervisorctl reread
		sudo supervisorctl update
		sudo supervisorctl start worker:*

Series Author:  Thomas Damsgaard

Related articles