The power of Laravel Queues
Last updated: July 29th 2021
Introduction
While building your web application, you may have some tasks, that require more computer power or time than a typical web request can provide if you make the task run synchronously. If the user have to wait for a long time for the application to complete the task, it makes up for at bad user experience. The answer to this is to make the task asynchronous and run in the background while the user can procced their interaction with your application.
Laravel queues provide an API to use several queue backends. We will cover two options, eg. how to set up Redis and a relational database, that both comes out of the box on our Perfect Server Stacks. You can choose to use one or both of them in your application.
In this article we will build a simple queue system for our fictional freemium SAAS application, where we will send the new user a notification/email 5 days after registration, trying to onboard them to the paid service.
Prerequisites
Spin up one of our perfect servers. You will also need an understanding of Laravel observers and emails.
Key concepts
- Jobs - tasks that stack on to one or more queues.
- Workers - processes the job. If no worker is being active, no jobs will be proccesed. An important thing during development is to remember to restart your worker everytime you change a job. If not done, your changes won't take effect. Do it by typing
php artisan queue:restart
into the console or you can typephp artisan queue:listen
as an alternative toqueue:work
that lets you watch the queue process so you don’t have to manually restart the worker after code changes. - Serialization - detects the eloquent model and serializes the models attributes for the job to use. Also deserializes the model for the worker to handle the jobs logic. Happens automatically in the Laravel pipeline.
- Daemons - means that the worker you set up to execute your tasks is stored in memory, thus not reflecting changes you make to your code before restarted. Ultimatly you will need to restart the worker every time you deploy new code to your production server.
- Delayed job execution -
delay()
helper function. Specific to the functionallity of this article example.
Application preparation
Follow along to get some boilerplate code to work with.
First, install the Laravel Breeze package and modify the users migration to determine if they are subscribed or not. To do so add a subscribed boolean as follows - file path: database/migrations/create_users_table. Addition marked in bold
public function up() { Schema::create('users', function (Blueprint $table) { $table->id(); $table->string('name'); $table->string('email')->unique(); $table->timestamp('email_verified_at')->nullable(); $table->string('password'); $table->boolean('subscribed')->default(false); $table->rememberToken(); $table->timestamps(); }); }
Migrate the migration files
php artisan migrate
Next up, generate a mail class in the console
php artisan make:mail UserOnboarding
Build the mail - file path: app/Mail/UserOnboarding.
public function build() { return $this->from('example@webdock.io') ->subject('Queued Email') ->view('emails.user.onboarding'); }
Then create a new blade template to hold the mail view, we called it onboarding.blade.php and placed it in an user/emails subfolder. Modify the email template to your liking - file path: resources/views/emails/user/onboarding.blade.php
<!DOCTYPE html> <html> <head> <title>Webdock Laravel Queue</title> </head> <body> <p>Test email.</p> </body> </html>
Create and set up the job class for the registration onboarding process. The console command will generate a new folder in the app directory. In the console, type:
php artisan make:job UserOnboardingEmail
Typehint the user in the constructor to leverage Laravels route-model binding and in the handle method you will specify the user and the email class - file path: app/Jobs/UserOnboardingEmail.
class UserOnboardingEmail implements ShouldQueue { protected $user; public function __construct(User $user) { $this->user = $user; } public function handle() { // Check if the user is paying if (!$this->user->subscribed) { Mail::to($this->user->email)->send(new UserOnboarding($this->user)); } } }
Set up an event listener if this job is the only thing to listen for in the user model. If more actions are required, use an observer class if you later want to listen for many events on a given model to group all of your listeners into a single class. We will do the latter. This console command will also generate a new subfolder in the app directory for you.
php artisan make:observer UserObserver --model=User
Instantiate the UserOnboardingEmail job in the UserObserver class. Notice the delay()
function that will ensure the email is sent 5 days after registration - file path: app/Observers/UserObserver
class UserObserver { public function created(User $user) { UserOnboardingEmail::dispatch($user)->delay(now()->addDays(5)); } }
And registrer in the EventServiceProvider - file path: app/Providers/EventServiceProvider.
public function boot() { User::observe(UserObserver::class); }
Failed jobs
Sometimes jobs will fail. After a job fails it will be inserted into the failed_jobs database table which comes out off the box with your Laravel installation.
To retry failed jobs, add the all flag to the queue:retry
command
php artisan queue:retry all
To delete all of your failed jobs from the failed_jobs table, use the queue:flush
command
php artisan queue:flush
Excecuting jobs using Redis
Setting up Redis
On our perfect stacks, Redis is already installed so setting it up is simple. First install the redis composer package via the console and then locate the .env file and change the QUEUE_CONNECTION to redis.
Console
composer require predis/predis
.env file
QUEUE_CONNECTION=redis
Running php artisan queue:work
in the console should now be enough.
Excecuting jobs using a relational database
In order to use the database queue driver, you will need a database table to hold the jobs. To generate a migration that creates this table, run the queue:table
Artisan command. Once the migration has been created, you may migrate your database using the migrate command. Finish by changing the QUEUE_CONNECTION in your environment file.
Console
php artisan queue:table php artisan migrate
.env
# .env QUEUE_CONNECTION=database
Running php artisan queue:work
should do the trick.
Keep the queue worker running on a production server
In production, a queue:work process may stop running if you close the console, exceeds worker timeout or restart the worker. To handle this situation you can configure a process monitor that watch your worker(s) and restart them if they exit for some of the before mentioned reasons. Supervisor is our favorite tool and the one recommended by Laravel itself. Get it by typing the following in the console:
sudo apt-get install supervisor
Supervisor configuration files are typically stored in the /etc/supervisor/conf.d directory. Create a configuration file for Supervisor to monitor the workers. We call it worker.conf:
cd /etc/supervisor/conf.d touch worker.conf
And fill it as follows:
[program:worker] process_name=%(program_name)s_%(process_num)02d command=php /path/to/your/laravel/installation/artisan queue:work autostart=true autorestart=true stopasgroup=true killasgroup=true user=youruser numprocs=1 redirect_stderr=true stdout_logfile=/path/to/your/laravel/installation/worker.log stopwaitsecs=3600
For more information about Supervisor and it's functionallity, consult the Supervisor documentation.
Once the configuration file has been created, update the Supervisor configuration and start the processes, again from the console:
sudo supervisorctl reread sudo supervisorctl update sudo supervisorctl start worker:*
Series Author: Thomas Damsgaard
Related articles
-
A production workflow for Laravel using Webdock servers
-
Laravel development server with aaPanel on Webdock VPS for beginners
-
Laravel development server with aaPanel on Webdock VPS for beginners - Part 2
-
Multiple Laravel installs in subfolders - Nginx rewrite rules, full guide
-
Laravel Chunked Upload - uploading HUGE files
-
Laravel and Redis 5 - Introduction, Installation and basic use
-
Laravel and Redis 5 - A Functional Example
-
Build an API in Laravel with JSON Web Tokens (JWT) (Part 1)
-
The Anatomy of JSON Web Tokens (JWT) (Part 2)
-
Database and tracking JSON Web Tokens (JWT) (Part 3)
In this article we will build a database and track our JSON Web Tokens.
-
JSON Web Tokens and Laravel API custom filtering (JWT) (Part 4)