CiviCRM Community Forums (archive)

http://paste2.org/p/2727676

This is what I'm trying without good results.

1. Paypal Pro 'non-recurring' payments return less information compared to Paypal Standard and Paypal Pro recurring payments.  If you look at the Paypal documentation you can see that AMT (amount) is the only variable that is returned, wheras Paypal standard returns both a total and a fee.    This explains why fees have not been getting recorded with Paypal Pro transactions.   There is nothing we can change about the limited information Paypal chooses to return to us when the transaction is processed...but...

2. We can enhance CiviCRM's automated tasks to use the 'cron' to check back in with Paypal later.  It works like this:
    a. each hour or whatever, use CiviCRM's cron to ask paypal for information on contributions and give the transcation id#
    b. ask paypal for the fee amount
    c. go back retroactively to contributions with a blank fee, and "fill in the blank"

This is not a trivial task, but it's very doable according to Lobo.

He estimates between 5 and 15 hours of core team development work to make this a reality. 

Stoob or Lobo - Maybe I'm being obtuse, but it seems to me that the whole cron approach is needlessly elaborate when CiviCRM is not only already being sent the appropriate data via the IPN but is also already using said IPN to update the contribution record (flagging the transaction as complete, rather than pending), etc.

Can either of you please explain why we would even try to get the fee data from the API instead of the IPN? That seems like a much more complicated (and time consuming) way to do it...

EDIT - Someone is thinking ahead of me/us, there is already code in the IPN handling process that *should* be doing exactly what needs to be happening with fees and net amounts. So, the question is, why isn't it working?

Okay, two attempts down, neither successful. Possibly time for someone with more experience coding for the platform to take over, or at least give me some guidance. I'm working on an install of 4.2.4, if it matters.

My first attempt was to add the following to core/payments/PayPalProIPN.php around line 265 (in function single).

$contribution->fee_amount = $input['fee_amount'];
$contribution->net_amount = $input['amount'] - $input['fee_amount'];
When that didn't work, I looked a little closer and realized that:

a) What I wrote above more or less duplicates the following from function completeTransaction around line 427 from core/payments/BaseIPN.php

    if (CRM_Utils_Array::value('net_amount', $input, 0) == 0 &&
      CRM_Utils_Array::value('fee_amount', $input, 0) != 0
    ) {
      $input['net_amount'] = $input['amount'] - $input['fee_amount'];
    }
b) function getInput in core/payments/PayPalProIPN.php calls a nonexistent IPN value "settle_amount" around line 379, which I figured might be somehow producing a non-zero value that was preventing the above code from firing properly. In fact, there is no IPN value for net amount, so I replaced:

$input['net_amount'] = self::retrieve('settle_amount', 'Money', 'POST', FALSE);
with

$input['net_amount'] = 0;
This also didn't work. So... I'm left with two theories at the moment:

1. Due to my unfamiliarity with CiviCRM conventions, I've made mistakes with both attempts that prevent them from working despite being valid concepts.
2. $input['fee_amount'] is not getting set properly, or is not being properly passed to  function completeTransaction in core/payments/BaseIPN.php. I have no evidence of this (and I'm not sure how to log $input['fee_amount']'s value in the completeTransaction function - help?), but it would explain why the existing code that SHOULD be calculating the net amount from the fee and the amount and recording both is not doing so.

In conclusion, someone who knows what they're doing here more than I do, help/guidance please?

Granddesigns, I admire your ambition, but I think you may have misunderstood my previous post about the findings of Lobo and myself.  Let me make it super clear: the IPN for PayPalPro does not return the fee amount.   This was the result of our testing. 

You may not believe that, but that's why I included the link to the PaypalPro documentation.

While I would love to be mistaken and be proven wrong, I see from your posts that you are assuming that PaypalPro does return the fee in the IPN as PayPal Standard does.   But it doesn't.  That's the problem.

Stoob, you will be happy to hear that you and Lobo are indeed mistaken. PaypalPro returns fee information as part of the IPN, just as PayPal Standard does. The documentation you linked to pertains only to information returned via the Website Payments Pro NVP API, which is NOT the same thing.  Here is the correct documentation for you to be looking at.

To further illustrate this, I have included below the complete text of the IPNs from the last three (non-express) donations made through my live site, omitting only identifying details (replaced with ****). You will notice that the transaction fee is included as both mc_fee AND as payment_fee (though according to the documentation, mc_fee is the correct value to use).

mc_gross=50.00&invoice=****&protection_eligibility=Ineligible&payer_id=****&tax=0.00&payment_date=10:52:13 Jan 19, 2013 PST&payment_status=Completed&charset=windows-1252&first_name=****&mc_fee=1.40&notify_version=3.7&custom=&payer_status=unverified&business=****&quantity=1&verify_sign=****&payer_email=****&txn_id=****&payment_type=instant&last_name=****&receiver_email=****&payment_fee=1.40&receiver_id=****&txn_type=web_accept&item_name=Online Contribution: Give a few dollars. Make a big difference.&mc_currency=USD&item_number=&residence_country=US&receipt_id=****&handling_amount=0.00&transaction_subject=&payment_gross=50.00&shipping=0.00&ipn_track_id=****

mc_gross=25.00&invoice=****&protection_eligibility=Ineligible&payer_id=****&tax=0.00&payment_date=09:06:26 Jan 19, 2013 PST&payment_status=Completed&charset=windows-1252&first_name=****&mc_fee=0.85&notify_version=3.7&custom=&payer_status=unverified&business=****&quantity=1&verify_sign=****&payer_email=****&txn_id=****&payment_type=instant&last_name=****&receiver_email=****&payment_fee=0.85&receiver_id=****&txn_type=web_accept&item_name=Online Contribution: Give a few dollars. Make a big difference.&mc_currency=USD&item_number=&residence_country=US&receipt_id=****&handling_amount=0.00&transaction_subject=&payment_gross=25.00&shipping=0.00&ipn_track_id=****

mc_gross=1.00&invoice=****&protection_eligibility=Ineligible&payer_id=****&tax=0.00&payment_date=06:51:55 Jan 19, 2013 PST&payment_status=Completed&charset=windows-1252&first_name=****&mc_fee=0.32&notify_version=3.7&custom=&payer_status=unverified&business=****&quantity=1&verify_sign=****&payer_email=****&txn_id=****&payment_type=instant&last_name=****&receiver_email=****&payment_fee=0.32&receiver_id=****&txn_type=web_accept&item_name=Online Contribution: Give a few dollars. Make a big difference.&mc_currency=USD&item_number=&residence_country=US&receipt_id=****&handling_amount=0.00&transaction_subject=&payment_gross=1.00&shipping=0.00&ipn_track_id=****

I hope that proves the point beyond a reasonable doubt?

are we talking of a single payment via paypal pro or a recurring payment via paypal pro?

if the former, then civi does not have code that implement IPN for paypal pro. we use the doDirectPayment functionaltiy via NVP as u mentioned
e this
Based on your prior post, seems like it is. But civi is not expecting one and hence has not sent any info to paypal with regard to format of IPN (which include module name, contribution id etc) and hence we ignore it

bottom line, for civi to handle paypal pro ipn's someone will need to investigate this and submit a patch to take advantage of this

lobo

Fair enough, thanks.  Regardless of the approach, some effort is required by someone to create a patch.

Granddesigns, would you be kindly willing to financially sponsor the core team to create a patch and achieve this goal?  In the absence of directly created patches with your own hands, the project moves also moves forward with financial sponsorship for core team members to make patches.  I have financially sponsored many patches, although this one is not a priority for me personally.

Lobo's initial estimate was 5 to 15 hours.

i think the estimate remains the same. In general our lowest estimate is always 10 hours. In this case, whoever does it still needs to get upto speed on the latest changes by paypal etc and add unit tests to the whole scheme while they are at it

regarding your other issue, you should upgrade to 4.2.7 and see if it is resolved there.

lobo

would be great if you could invest your resources in debugging and fixing the issue with "no emails associated ..."

In the long run sticking to a specific point release might not be a great situation, IMO

lobo