Create Queue and Run Jobs Using worker in Lumen/Laravel

A queue is a very important and interesting feature of Lumen, You can run your tasks(background process/cron tasks) using queue without fail. You need to push tasks into the queue and the rest of the work will do by the worker.

Sometimes, We need to run a task one by one or partially which will process some data and return the final processed data as a response. You can use a queue in Lumen for periodically running tasks, cron tasks and background scripts tasks.

Normally, We are using CRON job to call task periodically but some time that task failed due to execution time, server response time 500, or any other server issue, next time that task will run on the scheduled time, again and again that process will continue but failed process will not retry if the scheduled task was failed.

I am using the following module for the Lumen queue

• Beanstalk: Beanstalk is a simple, fast work queue.It was designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

I will use the following file:

• Composer: I will use this file to add beanstalkd for install as a dependency module
• Config/Queue.php: This file will use for the configuration of Queue and his driver.
• Controller/sampleController.php: This controller file will use for the sent tasks to jobQueue
• Jobs/testQueue.php: This jobs file will responsible to push tasks into queue using the method.

You can also check other recommended tutorials of Lumen/Laravel,

• How to Monitor Beanstalkd Queue In Laravel Using Admin Console
• How to Configure supervisord on Linux for Laravel Jobs Queue
• Laravel Micro Rest Framework – Simple Example of RESTful API in Lumen
• How to Configure Memcached in Lumen Api Framework
• Simple Example of Laravel 5 Login System Using Sentry
• Authorization and Authentication of Users in laravel 5 Using Sentry
• Simple Laravel Layouts using Blade Template and Bootstrap Theme

Install Beanstalk on Linux server

sudo yum install beanstalkd
sudo /usr/bin/beanstalkd -l 127.0.0.1 -p 11300
sudo service beanstalkd start(stop/restart)

Step 1: Added beanstalkd dependency in composer.json, I am using beanstalkd 3.0+.
"pda/pheanstalk": "~3.0"

Now run update composer from the command line,
D:\lumen_code> composer update

Step 2: Added Queue configuration in config/queue.php file, If you do not have a cache folder then you need to create a Config folder in the root of the lumen application(ex. – lumen_code\Config) and copy queue.php file from vendor\laravel\lumen-framework\config and paste into lumen_code/config folder, so the new file location would be lumen_code/config/queue.php. You need update queue.php as like the below configuration.

'default' => 'beanstalkd',

'beanstalkd' => [
            'driver' => 'beanstalkd',
            'host'   => '127.0.0.1',
            'queue'  => 'default',
            'ttr'    => 3600,
        ],

We have changed the default queue driver to beanstalkd queue driver and updated beanstalkd driver configuration.

Step 3: Now I have created a new jobs file testQueue.php in jobs/ folder. We need to add the below code into testQueue.php file,

queue = $data['queue'];
    }
    /**
     * Execute the job.
     *
     * @param  
     * @return void
     */
    public function handle() {
      Log::info('Hello! Queue job '.$this->queue.' is run at start time - '.microtime(true));
 }
}

I have created parameterized queue method that will help to run tasks with parameters. I have just created a sample queue method which will print messages into LOG file, You can see into storage/Log/lumen.log file.

Step 4: We have successfully created jobs and created a new method that will push tasks into the queue, I have created test_queue() method into sampleController.php file like the below code which will push tasks into the queue.

use Illuminate\Support\Facades\Queue;
use App\Jobs\testQueue;

public function test_queue() {
 for($i=0; $i<=10; $i++){
        Queue::push(new testQueue(array('queue' => $i)));
  echo "successfully push";
    }
}

I have included testQueue which is earlier created for queue jobs.

How To run Laravel/Lumen jobs Queue Using Artisan

Artisan is the name of the command-line interface included with Laravel/Lumen. It provides a number of helpful commands for your use while developing your application. It is driven by the powerful Symfony Console component.

Run all Queued tasks in background

php artisan queue:listen &

Run single Queued task

php artisan queue:work

Conclusion

We have learned about Laravel/Lumen queue using Beanstalk. We have created a controller method that will push task into laravel queue and run the queued task using PHP Artisan from the command line, next time I will let you know how to schedule queued tasks background using the supervisor module.