Status "Pending Refund" when it should not in a Direct_Debit situation.

[magnolia61]

Hello,
We use direct debit (civisepa) as a payment processor (payment_type=2), which of course is a sort of pay later payment-processor, until we send the payments that are due to the bank.

Now we have a situation I think is a quirk in the partial-payments system.
1. A customer registered for an event using a price-set. Total sum: 150 euro
    (event registration and contribution get correctly recorded as pending)
2. The next day the customer emails and says she does need one of the items in the price set.
3. I change the payment, deselect the price set item.

What happens next is unexpected behaviour
a. The amount in the contribution changes correctly to the new total amount (total sum: 140 euro)
b. the contribution status changes to 'pending refund' (a refund of 140 euro?)
c. I cannot change the status back to 'pending from 'incomplete transaction'

It is the last part that is a problem for us. When handling direct-debit payments, contributions with status pending should keep their status after a price-set change.

A solution I would like to propose would be to handle direct-debit payments for events in the following way:
1. direct debit payments that directly transfer to a payment provider directly get the status 'in progress'.
2. direct debit payments that manually get processed get the status 'pending from incomplete transaction'. This status needs to change to 'in progress' when submitted to the bank.

Any help or wisdom on this suggestion or my original problem? The fact that the contribution gets the sum 140 and status 'pending refund' seems to be a bug, doesn't it? If I wouldn’t have noticed it we would have lost the contribution or refunded a huge amount in error.

Is there anyone out there with backend Direct_Debit experience? Any upcoming civicrm 4.7 changes I need to reckon with? Can we try to outline what would be expected behaviour for direct debit backend handling when price set selections are changed?

Please help me with some wisdom here and let's take direct debit into account in the process of maturing financial contributions in civicrm.

The issue I filed on this can be found at: https://issues.civicrm.org/jira/browse/CRM-16038

Topics that touch this subject:
http://forum.civicrm.org/index.php?topic=32411.30 (GSOC on refunds)
http://forum.civicrm.org/index.php/topic,32036.0.html (older topic on backend direct_debit support)
https://github.com/Project60/org.project60.sepa (Systopis payment extension for european SEPA direct debit)

[Eileen]

Richard,

We have been talking a LOT about this sort of thing over the past 2 weeks at CiviCon and the following sprint and have been trying to get ourselves in sync over this.

In general we would say that the payment processor has the option of setting 'contribution_status_id' =  during the doPayment function (doPayment is now the preferred function to call and by default calls doTransferPayment or doDirectPayment internally - this works in 4.6 & by 4.7 will hopefully be the only way the form layer calls the payment function).

A general concept we have been discussing is that we want to move towards a model where the contribution and associated entities are always created as pending objects and then the payment processor is called (if appropriate) and if it responds saying completed the transaction would be completed. This could happen in real time or later on. I guess in that model setting to 'in progress' would be something returned by the processor - but it would be functionally no different to pending.

[JoeMurray]

Lots of inter-related pieces here.

1. How to handle change to a pending at the processor transaction? We should assume it will go through, but what if it fails? It may get complicated to extricate ourselves.

2. It seems appropriate that a refund will be due for the difference between original total amount and new total amount. It's not clear from report if that is 10 or 140, but seems like it is 10.

3. If there is a refund due on an overpayment, I think the total on the contribution should show the amount paid after the refund is paid. So that would apparently be 140 here. We need the Refunds code still to be developed to support putting the amount to be refunded in a new area in the UI.

4. As the system is normally supposed to assume that a direct debit payment will proceed even though it may fail immediately due to incorrect account number or quicklu due to NSF or up to a couple months later due to payor declining to accept payment, I'm a bit unclear on when a refund for a submitted direct debit transaction should be submitted.

5. We should likely separate status of refund from of the original transacton to handle this case where the original is still pending and a refund is already due. We don't want to process the refund until the disposition of the original transaction as paid or failed is known.

6. Separate from all of this is the question of the direct debit workflows we will support. Reading between the lines, it seems that in this case the direct debit transaction had been entered in CiviCRM but not yet exported for submission to the payment processor. We should likely map out a separate status for this since CiviCRM can make the change without having to do a refund or worry about whether the transaction succeeds or not.

[Eileen]

"We should likely map out a separate status for this since CiviCRM can make the change without having to do a refund or worry about whether the transaction succeeds or not."

I think we really want the function that initiates the refund not to care whether the transaction succeeds or not. A function that can be called either immediately in the form flow based on the response, or later on based on an update can make all the required changes to reflect the outcome

[JoeMurray]

Eileen, I like your take on responsibilities for functions to maintain modularity and independence. But I'm still having trouble with business rules and showing statuses of everything going on to user so the unsophisticated one isn't confused and the expert one can do what they want. We generally have been assuming user knows what they want and getting out of their way. In this case it means there is a remote possibility that user receiving the refund could decline the transaction. But thinking this through, though the org would be out the refund due to allowing this workflow, the police could charge the malcreant.

[Eileen]

A refund failing (e.g the site being notified about the failure via IPN) could be due any number of 'oddities' - that I wouldn't even want to guess what they might be.

I think we need to assume that it could happen