How to set IPN callback for extensions?

[Chris Burgess]

I'm looking to bundle a processor we've been using for a while (Flo2Cash) as a pair of extensions (for three- and two-party versions).

But I don't see how IPN works yet. How do I provide an IPN callback from my extension? Will CiviCRM direct the IPN call to the extensions directory?

I've had a test with the Google Checkout extension (to enable which I had to remove the default version from the DB), but it doesn't seem to reference googleNotify.php in the extension code, and Google doesn't call the IPN (notify) file after I transact.

Pointers welcome!

Thanks

[Chris Burgess]

OK, so Google's IPN needs configuration in the merchant admin. Thanks Donald for the reminder

I might have missed something here ... first time I've played with extensions.

As far as I can tell, the CiviCRM extensions directory is intended to be writeable to your webserver, so that extensions can be retrieved and installed automatically. But the extensions dir may also contain IPN callback files, and this combination rings alarm bells for me - executable code in a webserver-writeable directory.

It means there's a place where any exploit of the webserver can store and execute code later. Generally I use Apache's sethandler to prevent execution in any directory which the webserver may write to for this reason.

I can live without CiviCRM being able to install extensions itself. But I wonder if we can find a way to do this which doesn't mean CiviCRM is encouraging writeable, executable directories.

If extensions can add their own callback paths to CiviCRM (eg: ?q=civicrm/googleNotify or ?q=civicrm/ipn/PayPal) then we can handle this in the extension. That way the extensions directory could include PHP classes only, and we can prevent exec entirely via sethandler.

[Eileen]

Hmm - I thought the key idea behind extensions was handling the extern folder. My understanding is that Lobo was keen to stop accepting payment processors into core & a cunning plan was hatched that would prevent us from whinging about them breaking every upgrade. The ipn files were always the main problem as the custom php directory was adequate for the other files.  I guess they still are

[Chris Burgess]

As I said ... it's possible I'm missing something. I had a go with the PayFlow extension as well, but it looked like it was going to take a bit of work before I'd be able to test out IPN there either.

Two ideas on how extensions might remove the need for adding separate scripts to the extern/ folder -

• Allow extensions to provide custom callbacks, eg ?q=civicrm/googleNotify style. This offers a lot more flexibility for extensions generally, and it's quite possible this is already available but I haven't found it yet.
• Create an IPN interface which payment processor extensions can implement (eg defined method like CRM_Core_Payment_myprocessor::handleNotify()), and have CiviCRM expose ?q=civicrm/notify which the processor would call with a parameter to specify the appropriate class to call that method in (?q=civicrm/notify&processor=myprocessor&blah=blah).

But if what you say is true, I expect there's already a means to work around the issue I'm having. And *maybe* it's one of the above! I'll go re-read the "implementing a payment processor extension" page again

[Chris Burgess]

There's a secondary issue with the current code, in that most of the IPN.php style implementations start with something like this -

Code: [Select]

<?php
session_start( );
require_once '../../../civicrm.config.php';
require_once 'CRM/Core/Config.php';

The above code won't work unless the extensions directory is in a specific location (inside the civicrm directory), placing googleNotify.php somewhere like modules/civicrm/[extensions]/org.civicrm.payment.googlecheckout/googleNotify.php - which means that a CiviCRM upgrade risks zapping the extensions folder.

If IPN callbacks are .php files in the extensions folder, those scripts each need to contain a method of discerning the correct CiviCRM module folder if the extensions folder is located anywhere else. Yuck.

If IPN callbacks are embedded in CiviCRM already (eg via one of the previous suggested methods), this problem is solved.

[Donald Lobo]

u definitely should not place the extensions folder under modules/civicrm to avoid upgrade / delete issues

maybe the recommended place could be modules/civicrm.extensions

Since extension install / uninstall will not be too often, how about recommending that (if u have shell access)

1. People only make them web-writable when installing/un-installing an extension

2. if the extension directory is writable, we emit a warning on the admin page stating so

I think we might want to consider making the IPN part of the civicrm url structure (and this invoked via the CMS). This will solve the issue of: "where is the civicrm directory located etc"

lobo

[Chris Burgess]

CiviCRM warns if the extensions directory is not writeable for the webserver, and it looks like there is code to automatically retrieve extensions from extdir.civicrm.org. Requiring shell access to enable/disable easy installation of extensions seems like a step backwards - "here's a nice GUI, but before using it ..."

Agreed that nothing should go in modules/civicrm - I was flagging that the current org.civicrm.googlecheckout example processor seems to expect a specific location under there for googleNotify.php.

We generally install CiviCRM something like this -

Code: [Select]

sites/all/modules/contrib/civicrm/               base civicrm code, typically in 'contrib' subdir unless patched

sites/example.org/civicrm/
                          custom_tpl             site-customised code which may be executed, read-only for webserver
                          custom_php             

sites/example.org/files/civicrm/                 non-executable files, writeable for webserver
                                custom
                                persist
                                templates_c
                                upload

The idea behind this layout is:

• modules go in modules/ and are sorted into contrib, custom or patched
• only one directory, files, is writeable to the webserver, and it has execution disabled
• custom code lives in the site dir; it's executable so it's read-only for www

If CiviCRM wants to install extensions, which requires write access, then by the above model extensions go to files/civicrm for write access.

In the files directory, Drupal's SetHandler fix for SA-2006-006 will prevent .php files in extensions directories from being writeable.

I don't see a way we can site webserver-writeable, executable code in an accessible directory and not have it be a security concern (please do try to convince me if you think I'm wrong).

I believe we need CiviCRM to provide a means for extensions to expose IPN functionality. I've opened this as: http://issues.civicrm.org/jira/browse/CRM-8872

[Michał Mach]

Wow, tons of great thinking in this thread. :-)

From what I remember, original idea was to have an extern/ script, which would be always called by payment processor with a parameter identifying its type - so practically the same as you propose (not counting your improvement around IPN interface). I even thought it was actually implemented, but from quick look at the repository code, I don't see it anywhere. There were three of us digging around "extensioning" payment processors and it was during code sprint, so maybe this part slipped somewhere in between conversations/work division.

I would love to take a closer look at this as soon as possible, but I'm a bit behind with projects and need time until end of this week to catch up. Seems like I have raising queue of extensions issues (which means more people start using them, hurray) and plan to spend some uninterrupted time on fixing problems/introducing improvements at the beginning of next week. Part of that would be checking what's going on with IPN thingy and making it work.

Thx,
m

[Eileen]

Code: [Select]

Wow, tons of great thinking in this thread. :-)
Chris - I think you just got an endorsement for talking to yourself 

[Chris Burgess]

Thanks. I like to call it "peerless programming" when I'm yakking away to myself in public.

OK, so accepting that this not yet available in 3.4/4.0, I've done a couple of things -

In Flo2CashDonate class, I've added a paymentNotify() method which is really just a wrapper for Flo2CashDonateIPN::main(). I think we want to have this go via the processor, so CiviCRM has a clean interface against the payment processor rather than expecting a second file with magic 'IPN' suffix.

I'd love to talk further about whether we can / should standardise such a method for payment processors to implement. Eileen I think I saw in PayFlow that you had different notification handler for single and recurrent contribs, for example?

Here's what I've kicked off with anyway - a two-line wrapper

https://github.com/GiantRobot/nz.co.giantrobot.flo2cashdonate/blob/master/Flo2CashDonate.php#L276

Also, in the processor's __construct(), I added a check to see if the notify handler has been copied to extern/ ... This means admins will get a pointer in the right direction, at least until we handle this better. I was expecting this would trigger a warning on the payment processor notification page, but seems it only does it on the actual contribution page (and again in demo mode). Warning is for admin eyes only.

https://github.com/GiantRobot/nz.co.giantrobot.flo2cashdonate/blob/master/Flo2CashDonate.php#L81

Keen to discuss the best way to add a handler eg: civicrm/paymentnotify which we can use to fire the paymentNotify() method in the relevant extension as well.

[Eileen]

If you stuck a generic paymentnotify handler in the extern how would it know which processor type was calling it - they are all quite different - the buggers.

[Chris Burgess]

"two ideas" here: http://forum.civicrm.org/index.php/topic,21543.msg90361.html#msg90361

Then sll the different parts sit inside ::paymentnotify()

[Michał Mach]

If you stuck a generic paymentnotify handler in the extern how would it know which processor type was calling it - they are all quite different - the buggers.

I think we'll go with Grobot's second proposal - unless PPs cannot handle callback url with params? (doubt it) - however, I do feel tempted to investigate custom callback option... :-)

Do you guys want to have quick IRC chat on Monday? :-)

m

[Chris Burgess]

If someone has their heart set on combining a hosting environment without mod_rewrite/clean url support *and* a payment processor which doesn't handle params, they will need to use something like this f2c_ipn.php which is just a file wrapper for something::paymentNotify() anyway.

[RobertPattinson]

I can live without CiviCRM being able to install extensions itself. But I wonder if we can find a way to do this which doesn't mean CiviCRM is encouraging writeable, executable directories.