CiviCRM Community Forums (archive)

There's an extension called "org.civicrm.demoqueue" which may be easier to read than the others:

https://github.com/civicrm/civicrm-core/tree/master/tools/extensions/org.civicrm.demoqueue
https://github.com/civicrm/civicrm-core/tree/master/tools/extensions/org.civicrm.demoqueue/CRM/Demoqueue/Page

You've probably figured this out already, but the main concepts are:
 * Task: A particular unit of work. (Note: The queue can store anything serializable, but if you want to use the QueueRunner, then the queue should store Task objects.)
 * Queue: The stored list of tasks. This usually stored in a SQL table (CRM_Queue_Queue_Sql).
 * QueueRunner: The worker which consumes the tasks iteratively, executing each. For CLI usage (like drush), one would call runAll() or runNext(). For the web UI, one would call $runner->runAllViaWeb(). runAllViaWeb is a terminal call - executing it will immediately trigger a redirect to the queue UI.

There's a little more documentation about the options accepted by QueueRunner in the docblocks:

https://github.com/civicrm/civicrm-core/blob/master/CRM/Queue/Runner.php#L90

I know that this post is a bit old though I am still having the same question. What about documentation and tutorial on using the CiviCRM Queue system? Is that available and where do I find it?

Cool! Thanks, Jaap.

== Suggestion #1: Interactive vs background use-case ==

There are two steps here:
 * Step 3 create a page which executes all tasks in the queue
 * Step 7 create a cron job

These seem like two alternative user-experiences to me -- you'd pick one depending on the use-case. If you want to run an interactive import and display a progress bar, then you'd do something like step 3. If instead you wanted to defer a task so that it runs in the background (headless), then you'd setup a cron job. So maybe it would make step 3 would be more like:

 * Step 3: Decide how to execute the tasks
   + Step 3a: Execute the tasks interactively (web-based runner)
   + Step 3b: Execute the tasks in the background (cron-based runner)

== Suggestion #2: Structure vs Indirection ==

Comparing the wiki tutorial with org.civicrm.demoqueue, they take a different trade-off between being "structured" vs "simple". The tutorial (4-5 files, 11-12 functions) puts in a lot of structure but adds indirection and boilerplate which makes it harder to trace; the demoqueue (2 files, 6 functions; (https://github.com/civicrm/civicrm-core/blob/d7c8cf03e9d/tools/extensions/org.civicrm.demoqueue/CRM/Demoqueue/Page) is the opposite.

The greater structure seems useful in developing a generic/multi-purpose tool (like CiviRules), but it's more complex than I would want to write in a typical case. The demoqueue has a similar fault -- it's too simple; real code would be more structured.

How would about splitting responsibilities like this: https://gist.github.com/totten/dd9e9f02bc5ed68597db

(Random aside; the demo-queue uses CRM_Core_Page and sets up the queue in run(). I think for an extension author, they're more likely to use CRM_Core_Form and set up the queue in postProcess(). I was just too lazy to write out that example.)