How to create Queue and Run Jobs using worker in Lumen/Laravel

Queue is very important and interesting features of Lumen,You can run your tasks(background process/cron tasks) using queue without fail.You pushed task into queue and rest of work will do by worker.Sometime we need to run a task one by one or partially which will process some data and return final processed data as a response.You can use queue in Lumen for periodically running tasks,cron tasks and background scripts tasks.

Normally we are using CRON job to call task periodically but some time that task failed due execution time,server response time 500 or any other server issue,next time that task will run on scheduled time, again and again that process will continue but failed process will not retry if scheduled task was failed.

I am using following module for Lumen queue

  • Beanstalk : Beanstalk is a simple, fast work queue.It was designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

I will use following file:

  • Composer : I will use this file to add beanstalkd for install as a dependency module
  • Config/Queue.php : This file will use for configuration of Queue and hist driver.
  • Controller/sampleController.php : This controller file will use for sent task to jobQueue
  • Jobs/testQueue.php : This jobs file will responsible to push task into queue using method.

You can also check other recommended tutorials of Lumen/Laravel,

Install Beanstalk on Linux server

Select Code
1
2
3
sudo yum install beanstalkd
sudo /usr/bin/beanstalkd -l 127.0.0.1 -p 11300
sudo service beanstalkd start(stop/restart)

Step 1: Added beanstalkd dependency in composer.json, I am using beanstalkd 3.0+.
"pda/pheanstalk": "~3.0"

Now run update composer from command line,
D:\lumen_code> composer update

Step 2: Added Queue configuration in config/queue.php file, If you do not have cache folder then you need create Config folder in root of limen application(ex. – lumen_code\Config) and copy queue.php file from vendor\laravel\lumen-framework\config and paste into lumen_code/config folder, so new file location would be lumen_code/config/queue.php. You need update queue.php as like below configuration.

Select Code
1
2
3
4
5
6
7
8
'default' => 'beanstalkd',

'beanstalkd' => [
            'driver' => 'beanstalkd',
            'host'   => '127.0.0.1',
            'queue'  => 'default',
            'ttr'    => 3600,
        ],

We have changed default queue driver to beanstalkd queue driver and updated beanstalkd driver configuration.

Step 3: Now I have created a new jobs file testQueue.php in jobs/ folder.We need to add below code into testQueue.php file,

Select Code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
<?php

namespace App\Jobs;

use App\Jobs\Job;
use Log;
use Illuminate\Http\Request;

class testQueue extends Job
{
    public $queue;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct($data)
    {
        $this->queue = $data['queue'];
    }
    /**
     * Execute the job.
     *
     * @param  
     * @return void
     */

    public function handle() {
      Log::info('Hello! Queue job '.$this->queue.' is run at start time - '.microtime(true));
 }
}

I have created parameterized queue method that will help to run task with parameter.I have just created sample queue method which will print messages into LOG file,You can see into storage/Log/lumen.log file.

Step 4: We have successfully created jobs and created a new method which will push task into queue,I have created test_queue() method into sampleController.php file like below code which will push tasks into queue.

Select Code
1
2
3
4
5
6
7
8
9
use Illuminate\Support\Facades\Queue;
use App\Jobs\testQueue;

public function test_queue() {
    for($i=0; $i<=10; $i++){
        Queue::push(new testQueue(array('queue' => $i)));
        echo "successfully push";
    }
}

I have included testQueue which is earlier created for queue jobs.

How To run Laravel/Lumen jobs Queue Using Artisan

Artisan is the name of the command-line interface included with Laravel/Lumen. It provides a number of helpful commands for your use while developing your application. It is driven by the powerful Symfony Console component.

Run all Queued task in bacakground

php artisan queue:listen &

Run single Queued task

php artisan queue:work

Conclusion

We have learn about Laravel/Lumen queue using Beanstalk.We have create controller method which will push task into laravel queue and run queued task using PHP Artisan from command line, nect time I will let you know how to schedule queued tasks background using supervisor module.