CiviCRM Community Forums (archive)

Hello Everyone,
                          I a female tech enthusiast interested in open source development, I have potential skills in the following technologies:php, javaScript, html and css, node.Js and I am familiar with Zapier and I heard about Gsoc on CiviCRM from a friend who seeks to implement this platform to solve local community problems since i was told its the 'best' CRM platform which most organization actually recommend

 I am aware of Zapiers ease of integration with many platforms such as Google sheets, MailChimp and why not supporting based on its popularity that CiviCRM be integrated too.

From my experience, Creating Triggers and Actions of CiviCRM on zapier has already been well simplified and from my perspective I think it can simply be done.

my main focus on this project is to adding CiviCRM API whcih can allow us to be able to makes such calls from CiviCRM which could do a Zap,Get and create Actions and Triggers and also how the Triggers and actions created on the Zap can be manipulated both on Zapier and on CiviCRM with the same data as data integrity is vital with such integration though that is handled by Zapier Already.

I request for the soonest possible help as i roundup with my proposal on this Gsoc Project.

Thanks for your soonest regards
Cheers,
Merci

Hi Erik,
             I will be carrying out today,

-- Re-install development environment ( Not Wp but Drupal)

-- Download and configure buildkit on environment

-- Explain the Authentication ( key/ api_key ) process on CiviCRM and the importance on auth in Zapier

 While using REST API, CiviCRM requires authentication for security purposes to return the data which was requested in the Get URL

 The same thing happens for Zapier,

    The Concept Here is that for every action which is created correspondingly in Zapier, We could as well use a get request to verify with the server on which the instance of CiviCRM is running and the security means be specified.

Specifically, The API key CiviCRM could be used in the get request to authenticate actually the API call of the user account which made the request and also the SITE_KEY in the civicrm.settings.php file in /drupal/sites/default directory will be used to identify the instance of the server where the get request is sent.

Locally I have tested that on my installation and this is how i did it.

- I created a new CM user specifying all the details

- I used the query to edit the the contact and add an API key to the newly created contact

" update civicrm_contact set api_key="123456789" where id = 203;"

- I obtained the site key from the civicrm.settings.php file

Using the above i modified the REST url of a Get Action to send the same request to be able to confirm that the authentication works perfectly

 Note:: without these two the server eventually returns an error message saying that authentication failed

The request now looks like this ::http://localhost/drupal/sites/all/modules/civicrm/extern/rest.php?entity=Contact&action=get&id=203&api_key=123456789&key=15df9066014102eb2d5edbc8002049d8&json={%22sequential%22:1}

and it return the contact details for the contact with id=203

NOW THE QUESTION IS HOW IS THIS APPLICABLE TO ZAPIER?

- Zapier makes use of the get requests to verify data and to communicate between servers and uses authentication methods such as API_key so if we choose and authentication method we make the request security ensured ( I concur )

- This same request can be specified under a trigger and then used to test for the returned data if data is successfully returned then Zapier both servers can communicate ( I am 60% sure it can happen )

- If this works then we may have to create a full trigger to test the validity of the get request and why not many API actions

- With  this done we can hence test actions by ensuring that the corresponding applications which have to react to this requests do exactly what they are expected

WHAT DO WE NEED TO START?

- Before Applying i had a free account which got expired before i could finish testing all of these: so i think a zapier account is needed ( i will do what you asked me @Erik)

- Access to online drupal instance of civiCRM to test request amongst servers online after testing the requests offline .

-  Install the API key extension on the online install, so as to generate api_keys for users for the requests

- Access to The site key for the standard drupal demo version for constant testing

WHAT WILL I DO NEXT?
 - I will contact the zapier helpdesk to request for an account for the testing 
 - While waiting on that i will propose the scope as a advice given me by the mentor

WORK ACCOMPLISHED

--Read the proposal to know how i can carve out the plan not to make it too deviated
-- revisited Time-line for Gsoc to be able to fit plan time period to avoid conflicts
-- created a draft plan on Google Docs
-- shared draft plan with mentor
-- prepared a few points for project planning and scope On Tuesday Morning

DIFFICULTIES

-- since i dont know the scope it was a little difficult to plan
-- dont yet know exactly how all the components will fit so tomorrow during discussion with mentor, i will ask all neccessary questions to be clarified

WHAT I DID YESTERDAY

-- Had a meeting with the mentor to plan the scope of the project

-- Reviewed user story 1

-- Re-planned user stories starting with user story 1

-- Created a new plan on Google Docs for planned user story and shared again with mentor

-- Read Zapier documentation
     * Getting started
     * Platform
     * App life cycle
     * Example App with OAuth authentication

WHAT I WILL DO TODAY

-- Read the dcumentaion on CONTRIBUTIONS on book.civicrm.org

-- Explored CiviContribute Module to come out with the vital information from the user story 1

-- Added Contributions on local installation using CiviContribute

-- used the ApI explorer to test for the presence of the Donation like that described in user story one ( returned some data)

-- Built sample GET request and tested on online CiviCRM for the same contribution ( http://civicoop.s700.sureserver.com/ehgsoc/sites/all/modules/civicrm/extern/rest.php?entity=Contribution&action=get&api_key=XXXXXXX&key=XXXXXXXXX&json={%22sequential%22:1,%22return%22:%22contact_id%22,%22contribution_status_id%22:%22Completed%22,%22total_amount%22:{%22%3E%22:50},%22financial_type_id%22:%22Donation%22,%22contact_id%22:203,%22api.Contact.get%22:{%22id%22:%22203%22}}&json=1&json=1

*** using the API key and the site key provided by the mentor 
*** the Json sample is here http://pastebin.com/m7W1jnbj

-- installed CiviRules extension as requested by mentor

-- verified get url on Zapier to test validity

PROBLEMS FACED

-- server returned a 403 error ( May be permission issues)

-- Contacted Zapier team, they took long to reply ( not as usual)

TO ADDRESS THESE DIFFICULTIES

-- i contacted the zapier help desk and explained the issues i faced

-- I pointed the team to the REST API documentation of  CiviCRM

-- Also I contacted the mentor telling him all what i have done so far

RECALL: As a fundraiser I want to automatically send a survey to every donor that donates 50 USD or more so that I am sure that every donor gets a survey

1. THIS HAPPENS IN CIVICRM

- A Contribution is entered, with all the necessary information ( donor , Financial type, Amount  etc)

## The API is used to query for the existing records on the contributions which meet the criteria specified in user story 1 ( donors with total amount >= 50 USD)##

- CiviRules will be used to get the data on through by querying the API for the the donations which meet the criteria for user story 1



- A rest API is created which will be used to send this same request.

- The result is obtained and then the contact_id(s) are obtained from the returned Json or XML result

- with this id(s) we can still do an api call now to get the contact information to get the data using the contact id(s) thereby helping us to know the donors by ( name , email , etc)

- An endpoint should be specified that will be used to provide this data to Zapier in a bundle ( may be Json or XML preferably) since Zapier will almost always convert JSon data into array format which is best understood

2.WHAT IS SENT TO ZAPIER

-  The bundled data ( contact details) is sent to Zapier  ( THIS SHOULD BE THE DATA REQUIRED TO CREATE A NEW SURVEY ON SURVEYPAL)

3. WHAT DOES ZAPIER DO WITH THAT

-The Authentication means is setup too ( probably API key of user requesting for resource )

- In zapier, A trigger is set with the Data source as a REST Webhook which will be used by Zapier to handle the subscriptions through CiviCRM REST API

- Zap is therefore configured with with SurveyPal by specifying both the CiviCRM account and the surveypal account

- The action is configured, to create a new survey with the data which was received from the REST Hook ( sort of )

-** Expecting that a new Survey be created, then should login to the survey pal account to verify the survey

** Zapier Does all of these with the help of its powerful REST Hooks

WHAT I DID TODAY


 ** Installed the CiviRules extension on the online install

 ** Read CiviRules Cookbook at https://wiki.civicrm.org/confluence/display/CRMDOC/CiviRules+Cookbook

  -  Wenr through the cookbook examples (

    CiviRules Basic Example Immediate Processing
    CiviRules Basic Example Scheduled Processing
    CiviRules Example Donor Classification
    Field Value Comparison Condition
)  on local install Detaily to understand how CiviRules extension can apply to user story 1

 ** Read on how Zapier support webhooks here https://zapier.com/help/how-get-started-webhooks-zapier/
 
  -Documented new findings and shared and shared with mentor ( will be posted tomorrow if deemed necessary )

FINDINGS
============

- Zapier understands data in these formats :


form-encoded   ---------  first_name=Bryan&last_name=Helmig&age=27

JSON ----
{
    "first_name": "Bryan",
    "last_name": "Helmig",
    "age": 27
}


XML ----

<first_name>Bryan</first_name>
<last_name>Helmig</last_name>
<age>27</age>


 HOW ZAPIER HANDLES WEBHOOKS
============================

When you create a new Zap with a webhook enabled trigger, a brand new URL is generated such as https://zapier.com/hooks/catch/n/Lx2RH/

This URL is provided to a triggering APP ( CiviCRM  in our case ). with this zapier should start receiving payloads as soon as the triggering app sends them

Note: Zapier also has the possibility of doing the reverse :: A URL is provided to zapier where it pushes the payloads so that CiviCRM can understand



QUESTION:: DOES CIVICRM IMPLEMENT WEBHOOKS AND HOW IS THAT DONE ?


GENERATING YOU OWN REQUESTS
===========================
This section entails triggering zapier webhooks without involving a third party app  (( this is done with the help of  "curl"  and "hurl" )  hurl is foudn here https://www.hurl.it/

 This comprises sending a post request with JSON payload

EXAMPLE using curl

curl -v -H "Accept: application/json" \
        -H "Content-Type: application/json" \
        -X POST \
        -d '{"first_name":"Bryan","last_name":"Helmig","age":27}' \
        https://zapier.com/hooks/catch/n/Lx2RH/

HOW DO WE KNOW THAT THE REQUESTS ARE SENT?
=======================================

A app like " Request bin "  here (http://requestb.in) which just takes the request and prints the payload content to the screen for us to verify that the request works actually

Therefor webhooks should be made available for the data set we require

CATHING WEBHOOKS
=================

This has to do with how we get payload from the webhooks sent by Zapier So this is also a vital step

for example if the payload type is json, it could be handled like this

<?php
$data = json_decode(file_get_contents('php://input'));
echo $data['path_to_file'];
echo $data['file_size'];
echo $data['hello'];
?>