CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • Google Summer of Code »
  • Email Preview Cluster
Pages: 1 2 [3]

Author Topic: Email Preview Cluster  (Read 5384 times)

totten

  • Administrator
  • Ask me questions
  • *****
  • Posts: 695
  • Karma: 64
Re: Email Preview Cluster
June 25, 2015, 12:40:58 am
I've posted the skeleton for the preview manager: https://github.com/civicrm/prevem You should have permission to clone/modify that repository.

The README has instructions on downloading and starting. Let me know if they work.

The README also includes some examples of how I think you could interact with it. LoopBack automatically generates a whole bunch of APIs ("scaffolding"), and most of the examples are based on these. You can learn more about them by playing with the built-in API "Explorer" and by reading http://docs.strongloop.com/display/public/LB/Working+with+data

There are two operations which need to deviate from the scaffolding:

  • When the Composer requests a new batch of previews, it gives a list of "renderers" to use. Based on this list, we have to create a series of separate "PreviewTasks" (one for each renderer). I followed the directions from http://docs.strongloop.com/display/public/LB/Operation+hooks and did this part. ( https://github.com/civicrm/prevem/blob/master/common/models/preview-batch.js )
  • When a Renderer requests a task, we need to search for a task which (a) has a matching "renderer" and (b) is not already been worked on. Then (c) we need to reserve that task so that other Renderers won't work on it. This will require writing a custom method (eg "POST /api/PreviewTasks/claim"). There's some documentation about that at http://docs.strongloop.com/display/public/LB/Remote+methods , but this still needs to be done.

The skeleton has a few other issues which are less urgent. I've posted these to the issue queue -- https://github.com/civicrm/prevem/issues

utkarshsharma

  • I’m new here
  • *
  • Posts: 22
  • Karma: 0
  • CiviCRM version: 4.5.8
  • CMS version: Drupal 7
  • MySQL version: 5.5.41
  • PHP version: 5.5.9
Re: Email Preview Cluster
July 09, 2015, 01:33:45 pm
The claim thing is done. I've created a pull request. Just check if this is what we were trying to do.
I'll immediately start work on the rest of the things we talked about the other day.

utkarshsharma

  • I’m new here
  • *
  • Posts: 22
  • Karma: 0
  • CiviCRM version: 4.5.8
  • CMS version: Drupal 7
  • MySQL version: 5.5.41
  • PHP version: 5.5.9
Re: Email Preview Cluster
July 19, 2015, 03:42:55 pm
The renderer can now claim a task and post the base64 encoded image back to the the particular task. Everything between the manager and the renderer seems to be working fine. I'll push the changes to github by tomorrow maybe after performing some final tests.
I'll just list down what all is happening rightnow:
1. Renderer polls the manager.
2a. If there is a pending task, it takes the screenshot, base64-encodes it and posts it to the "result" field of the task. Waits 10 seconds before polling the manager again. (Should I remove this wait?)
2b. If there are no pending tasks, it will wait 60 seconds before polling the manager again.

I also decoded the base64 string back to check if it was working properly and it was.
The expire time (Task_TTL) has been set to 5 minutes.
Now with this done, is there something else that needs to be done on the renderer side before we can move on to the composer side?

On a different note:
The code's getting bigger and it might be getting difficult to understand. I will need to take some time out to properly comment and indent it.
Also, in almost the entire code, I haven't placed error handlers. Especially the webdriverio part, because it keeps getting errors due to slow internet and the entire renderer process gets killed. I'll need to do that as well.
At what stage in the project do you think I should do this?

utkarshsharma

  • I’m new here
  • *
  • Posts: 22
  • Karma: 0
  • CiviCRM version: 4.5.8
  • CMS version: Drupal 7
  • MySQL version: 5.5.41
  • PHP version: 5.5.9
Re: Email Preview Cluster
July 22, 2015, 10:29:15 am
Files pushed to git. Everything is working fine now, with gmail and yahoo both. Major changes are in the renderer main.js file. And the renderer and the manager can be seen to be working together using the loopback explorer.
Work on Composer side started:
preview-batch.js has to be changed in a way to include a function that will tell the composer of it's batch's status. GET should be used here.
preview-task.js has to be changed in a way to let the preview-batch know when a task has been picked/completed. POST should be used here.

totten

  • Administrator
  • Ask me questions
  • *****
  • Posts: 695
  • Karma: 64
Re: Email Preview Cluster
August 20, 2015, 03:01:34 am
Just in case anyone comes back to look at this thread and wonders what's been going on... as we got into the final month of work, Utkarsh and I started doing more frequent status check-ins on IRC. There's been a lot of progress -- e.g. a big milestone was reached a couple weeks ago with a working roundtrip (with all the components talking to each other), but there's also been a lot of improvements on https://github.com/civicrm/prevem/issues

Anyway, keep an eye on the blog in the next few days. I think we'll be getting a proper report. :)

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Email Preview Cluster
August 23, 2015, 02:18:34 am
glad to hear

is there something we can test already?
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

utkarshsharma

  • I’m new here
  • *
  • Posts: 22
  • Karma: 0
  • CiviCRM version: 4.5.8
  • CMS version: Drupal 7
  • MySQL version: 5.5.41
  • PHP version: 5.5.9
Re: Email Preview Cluster
August 23, 2015, 02:35:20 am
Hi xavier!
I've already created a PR and we're hoping to get it passed through the tests so we can start beta testing. Tim will tell you more about it. Maybe you can check the blog post I made about the project once. I'd love to hear the community's feedback. Here's the link:
https://civicrm.org/blogs/utkarshsharma/email-preview-cluster-gsoc-completion-blog

Pages: 1 2 [3]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • Google Summer of Code »
  • Email Preview Cluster

This forum was archived on 2017-11-26.