[GSoC 2016] CiviREBUX: Report Building Module Extension for CiviCRM

[ypranay]

Hi All,

I am Pranay. I am a fourth-year Computer Science undergrad, pursuing Bachelors and Masters (Dual Degree) from IIT Kharagpur, India. I heard about CiviCRM from my friend, Utkarsh Sharma, who worked with CiviCRM during last year's GSoC and worked closely with him on his project. I really liked the concept of the project he was working on and that's what motivated me to work on a project this year with CiviCRM through GSoC.

The project titled Report Builder GUI caught my eye particularly because I had worked on a project very similar to this one during my internship at Symantec right after my second year. I worked with the Telemetry team on building a web dashboard for the enterprise clients of Symantec's prime security product - Symantec Endpoint Protection (SEP), where they can request, view and download the SEP Performance reports based on the filters set by them viz. period for the report (daily, weekly, monthly or custom CRON). Entire code was reviewed, perfected and pushed to production. It was during this time, when I got myself introduced with JasperReports and several other modules like JFreeChart etc.

I see that the project Report Builder GUI intersects to a great extent with the one I did during my internship, hence the application. As far as my professional experience is concerned, I have worked with several big names in the Software Industry viz. GE and Symantec as a Software Engineering Intern. I recently interned with Hike, India's fastest growing messaging app with over 100M mobile users in India and over 40B messages per month, as a Machine Learning Intern.

I have been in contact with Brian and Jaap, the mentors associated with this project, since past few weeks and they have helped me to a great extent in identifying exactly what areas should be focused on and possible approaches for the project. I will start working on the proposal soon! Can't wait to get started !! 

[ypranay]

Hi Everyone!

It feels great to be a part of the dev community at CiviCRM this year through GSoC. Special thanks to my mentors - Brian and Jaap, for all their support and guidance so far. Hoping to deliver my best this summer and contribute significantly towards CiviCRM.

I am having my end-semester exams now followed by thesis evaluation, so will be off for a couple of weeks till 7th May. I will mostly be available through Email - y.pranay@gmail.com and Skype [Search for y.pranay@outlook.com] for all this time and beyond. Please feel free to reach out to me anytime and I will try my best to be as quick as possible to get back to you 

I will be having my first Skype call with Brian and Jaap, after being selected for GSoC, this coming Tuesday to discuss the foundation of the project. Will get right back on track by mid-May, by cloning the latest CiviCRM code base and getting to understand how to clearly proceed within a few days time after that. Will keep the forum posted with any updates, experiences or even queries for everybody to see and contribute in any way they can.

Lets go FOSS this summer !!

P.S: Heartiest congratulations to the fellow GSoC students at CiviCRM this year!

[ypranay]

[Log: April 26, 2016]
Skype call with Brian and Jaap.
Minutes of Meeting:

• Weekly Check-ins via Skype on Wednesdays at 07:30 PM IST
• Daily Check-ins on the CiviCRM GSoC Forum
• Regular commits to the GitHub repo for the project (https://github.com/ypranay/ReportBuilderGUI)
• The proposed Report Building module will be constructed as a CiviCRM extension
• Setting up a Development website where Jaap and Brian can review and test my progress
• Reaching out to them via email or skype if I run into any issues or have questions, etc.
• Getting familiar with CiviCRM code-base and documentation
• Begin development full-time after May 9 - post my end-semester examinations and thesis evaluation

[Log: April 30, 2016]
TODO:

• Publishing a development website on my domain
• Get started with Civix: Extension Builder by totten

[ypranay]

[Log: May 09, 2016]

My domain space (given by the institute) is down since past couple of weeks. Requested Brian to help me with getting some dev  space where I can start working. Will revert back to mine, as soon as the former is up and running again.

DONE

• Got dev space from Brian last night.
• Installed WP

TODO for today

• Installing CiviCRM on this dev space
• Installing Civix for building module extension

AWAITING

• Publishing rights for writing a CiviCRM blog. Currently, disabled. Jaap has requested Michael McAndrew to provide access. Yet to hear back from him.

[ypranay]

Log: May 12, 2016

Still trying to fully install CiviCRM on the space provided by Brian. Resolved issues relating to ftp/ssh connections to the domain, installed WP and stuck at the final stage of CiviCRM installation.

First installed CiviCRM without checking the box for sample data. Decided to revert back to deleting CiviCRM and re-installing it. Facing some errors during re-installation. Contacted the mentors for some help regarding the same.

Will post the work-flow for the next week by tonight or early morning of May 13 in IST.

[ypranay]

[Log: May 13, 2016]

COMPLETED:

• CiviCRM, WP installation complete.
• SSH/FTP access to the dev space provided by Brian

ALMOST COMPLETED:

• Civix installation.
• Blog post for the past week.

IN THE PIPELINE FOR THE NEXT WEEK:

• Getting up to speed with AngularJS.
• Getting familiar with the jQuery UI plugins for building the proposed drag-and-drop interface for the extension.
• Basic layout of the extension.

[ypranay]

Log: May 14, 2016

COMPLETED:

• Civix installation
• Blog post live here - https://civicrm.org/blog/ypranay/gsoc-2016-repobuilder-report-builder-gui-for-civicrm

TODO FOR THE NEXT WEEK:

• Getting up to speed with AngularJS.
• Getting familiar with the jQuery UI plugins for building the proposed drag-and-drop interface for the extension.
• Basic layout of the extension using Civix.

[ypranay]

Log: May 19, 2016

With reference to the comment by lobo to my CiviCRM blog post here - https://civicrm.org/blog/ypranay/gsoc-2016-repobuilder-report-builder-gui-for-civicrm, I spent the day learning about modules and technologies that other reporting modules implement.

The ones which caught my eye particularly are GWT and Elastic Search. Although GWT seems cool, but I feel it would be difficult to actually use it to build the module, because of its inherent Java-based architecture. Elastic Search seems doable and can fit in at a later stage in the pipeline. As decided while writing the GSoC proposal, the module will have a AngularJS-powered front-end with a drag-and-drop interface built using jQuery UI plugins, just like a few others out there.

[ypranay]

Log: May 20, 2016

Spent the day finding myself around AngularJS documentation and tutorials here- https://docs.angularjs.org/tutorial. Built a small application on AngularJS having a simplistic GUI. Also,  followed the MVA Series on AngularJS- what, why and hows of AngularJS.

My personal favorite are the native AngularJS docs and tutorials. They are pretty comprehensive and explanatory for people who want to switch to AngularJS, like myself. A bit tough sometimes to comprehend, for which I would recommend seeing the MVA session on AngularJS, which we can find on YouTube pretty easily.

[ypranay]

[Log: June 1, 2016]

• Spent the day finishing up with the canvas built using Angular.js and gridster.js (http://gridster.net).
• Supports draggability, resizing, floating, pushing and swapping of blocks
• Doesn't support adding/deleting blocks as of now. User has to manually change the .js script
• Changed the repository and the name of the project to CiviREBUX (REport BUilding module eXtension for CiviCRM)
• Pushed the code to new GitHub repo here - https://github.com/ypranay/CiviREBUX

TODO:

• Add functionality for adding/deleting blocks dynamically.
• Getting started with using Civix for building Angular modules using civix generate:angular-module

[ypranay]

Hi All,

I am trying to see this extension - Activity Report (https://github.com/compucorp/uk.co.compucorp.civicrm.activityreport) into action. I downloaded the repository and placed it in the extensions directory of my WP site and installed the extension properly.

So far so good, when I click the "Activity Report" link under "Reports", I get a 500 error. I couldn't find any relevant error logs in the ConfigAndLog dir as well. I am a little clueless here. Any suggestions on how to resolve this error?

Thanks,
Pranay.

P.S: Resolved...

As Brian quoted, this extension is reported to be compatible with CiviCRM 4.2 and may not be accurate for CiviCRM 4.7. I looked at ConfigAndLog dir for useful logs but didn't find anything relevant. Looking at the error_log file within the wp-admin dir, revealed a reference to a Drupal-specific translate function t() in the Data.php file. Deleting the function call ultimately solved the issue.

[ypranay]

It's been quite some days since my last update. Sorry for that. Here is what I have been doing for the past couple of weeks:

[Log: June 10-June 17]

• Used civix to generate the module extension directory
• Added module page for generating contribution reports within CiviCRM
• Understanding the CiviCRM v3 API for Contribution data from here: https://github.com/civicrm/civicrm-core/tree/master/api/v3/examples/Contribution
• Implementing Pivottable.js with the Contribution data

Spent the initial few days trying to build angular modules for the extension, but discontinued. Since, most of the proposed drag-n-drop interface and the data transformation operations would be taken care of by the Pivottable.js jQuery library alone, using Angular would have been an overkill, given that I am new to Angular.

[Log: June 18-June 21]

• Added navigation link within the CiviCRM dashboard for the extension: Reports >> CiviREBUX
• Finished bug-fixing; Pivottable.js works well for the Contribution data
• Added a few more derived attributes, apart from the default attributes for Contribution data

Faced a few difficulties while adding the navigation link for extension. This extension by Jamie and others: https://github.com/compucorp/uk.co.compucorp.civicrm.activityreport was a great help throughout past two weeks. The code is well documented and was easy enough to follow. Thanks to @jamienovick for introducing this extension to me.

Here is the link to the updated GitHub repository for my module extension: https://github.com/ypranay/org.civicrm.civirebux. Please feel free to install the extension!

For the next few days, I will look at how to export the contents of the pivot table (including C3-rendered graphical pivot charts), so that it can be embedded inside a report. I will also speak to my mentors, who have been a great help throughout the past 2 weeks, for helping me decide as to what I should be doing for the next couple of days before the Mid-term Evaluation deadline.

[ypranay]

[Log: June 22, 2016]
Today and in the next few days, I will be focussing on the following things as per my discussion with Brian and Jaap yesterday:

• Adding support for excluding attributes from the Contribution summary
• Removing duplicate attributes from the Pivot table
• Include basic contact info attributes into the Pivot table
• Add "How To Use" banner on top of the extension page
• Add a few more useful derived attributes like quarterly contribution records

Currently, working on the second item in the above list. So far, no hack has worked. Will be spending few more hours on the same.

[ypranay]

Looks like I saved much time here. I am done with the following things (with reference to the previous post):

• Adding support for excluding attributes from the contribution summary: The important concern behind this issue was the presence of multiple duplicate attributes. This issue is now resolved. Additionally, the user can drag-n-drop attributes out of the scope of the pivot table as per his/her wish.   CHECK
• Removing duplicate attributes from the Pivot table: CHECK
• Include basic contact info attributes into the Pivot table: Added Sort Name, Display Name attributes. CHECK
• Add "How To Use" banner on top of the extension page:CHECK
• Add a few more useful derived attributes: Added Month-wise, Year-wise, Date-wise, Day-wise Receipts attributes. CHECK

The updated repository link is here: https://github.com/ypranay/org.civicrm.civirebux

[ypranay]

With reference to an email conversation between myself, Brian and Jaap a few hours ago, I added the following functionality as well:

• Added contact_type attribute to the contribution data pivot table
• Added a expand/minimise feature to the "How To Use" banner, in order to improve UX

Please refer to the extension repository for the latest code-base.