CiviCRM Community Forums (archive)

Context

On this forum, the idea was previously raised about reworking our form processing.  I've been involved in a few on-going conversations which touch on this from two very different angles.  Taken individually, each angle might lead to slightly different rethink of a form, but I'd like to brainstorm/talk-out-loud about how to make them work together.

Firstly, the folks at Giant Rabbit -- Kellie, Peter, Anna, Anthony -- have been talking about how administrators customize forms.  We currently have three techniques: (1) profiles in CiviContribute/CiviEvent/etc; (2) webform_civicrm; and (3) programmatic manipulation of core forms (using hooks, regions, jQuery, and/or Smarty-overrides).  The general hope is to combine the better parts of #1+#2, add missing parts, and make #3  simpler+rarer.  In both #1+#2, site admins pick and arrange fields on an individual basis, and I assumed (though not everyone assumed) that we would try to expose more of Civi for customization/arrangement in this fine-grained way.

Secondly, Eileen and Adam have been discussing the payment-processing system (eg https://civicrm.org/blogs/eileen/rethinking-payment-processing ).  In particular, Adam's working with a large non-profit that (a) uses a highly customized payment system and (b) works with many different (international) payment-processors.  In this case, the concern isn't about allowing an admin to pick-and-choose fields; rather, the concerns are (i) how to automatically adapt the form to meet the needs of different payment-processors and (ii) how to share this logic between civicrm-core and the customized system.  A compelling solution to this is to create a widget for the billing-block which adapts automatically and which can be embedded in different frontends.

My hope is that the two efforts can adopt compatible designs. As examples, here are some problems/conflicts to avoid:

• Suppose the form-builder gives the admin full freedom to pick/arrange all fields in a contribution page -- so he picks firstname, lastname, and credit-card number.  Now suppose that a payment-processor requires additional fields (street address, city, state, zip, country) that the admin hasn't included.  How does the field get displayed?
• Suppose the billing-block is implemented as a widget driven solely by metadata from the payment-processor -- e.g. the payment processor might say that it needs CC#, full-name, zip code, and country (but doesn't need city or street-address), So the billing-block includes exactly those fields --but omits street-address.  Now suppose an admin wants to collect street-address (because it's good to have on file), so he manually adds the field -- but it's not part of the billing-block.  Instead, it dangles on its own.

Proposal

The form system should till include support for adding/removing individual fields in a fine-grained way (as we currently do with profiles).  Additionally, it should support some bigger widgets.  These widgets try to combine fields in a sensible way. Some likely widgets would be:

• Name Widget: This widget is an optional mix of first-name, middle-name, last-name, prefix/suffix ("Dr", "Mrs", "Jr", "Sr").
• Address Widget: This widget is an optional mix of street-address, city, state, zip-code, country, county, latitude/longitude.
• Billing Widget: This widget is the billing block. It builds on the "Address Widget" (b/c it may have a mix of address fields) and introduces other fields (such as payment-processor-choice, CC#, or CVC).

I'm attaching three wireframes:

• FineGrained.png shows how one adds address fields in the fine-grained approach (akin to the existing profile functionality) -- basically, more like a drag/drop version of how profiles work.
• WithWidgets.png includes the same fields using an "Address" widget. Notably, the widget is given more prominence, but advanced users can drill down to "Core Fields" and manage each item individually. (If you look closely, it also shows the "Name" as a widget but doesn't go into as much detail about that.)
• BillingWidget.png shows a "Billing" widget which builds on top of the "Address" widget. It includes behavior to try to reconcile the preferences of the site-admin with the requirements of the payment-processor.

(Note: I was a bit lazy and basically copied/hacked some old wireframes from early discussions about Civi v4.3's form-builder for CiviSurvey, so... it looks a lot like that form-builder. There are other things going in this wireframe which can/should be debated -- but I'm just trying to highlight difference between fine-vs-medium-grained widgets.)

Pro/Con

• Pro: When building a form, it's easier to add a single "Address" widget (and set the location-type once) rather than searching through the list of fields for each field, adding it, and setting a separate location-type. You're also less likely to make mistakes due to inconsistent location-types.
• Pro: The default look/feel of a widget should be tighter and more "normal" than the look/feel of several separate fields. For example, with US addresses, it's conventional to put the city, state, and zip-code on the same line (in order).
• Con: This isn't Adobe PageMaker -- it doesn't give powerful layout. If you don't like the layout of the widget, the alternatives are to add fields individually, write an enhancement to the widget, or fund an enhancement to the widget.

Evolution

The layout of the name and address widgets may need some localization, and we'd probably want to include different locales by default.  Additionally, one might allow extensions to define or override widgets.

It may make sense to take some of the visual-language from the public forms of CiviContribute/CiviMember/CiviEvent (e.g. membership-levels, price-sets, premiums) and encapsulate these in widgets.  Personally, I'd vote for keeping these widgets close to the existing functionality at first but allowing them to evolve/be replaced/be extended over time.

Note: webform-civicrm (which has some constraints due to the nature of the webform module) takes the approach of adding all billing fields as a single block.
Upside: the system controls the fields inside the block so all the necessary ones will always be there.
Downside: the system controls the fields inside the block so the admin can't rearrange or style them.

Thank you, Tim, for putting those mockups together.  And Eileen for taking the the time to detail your work with Omnipay.  Would it be possible to initially add fields required by Payment Processors with the Billing Widget in your mockup, but then after they are added to the form, move the Address block above the Credit Card fields and the Email field below, for example?  I have not yet been asked to position City before Street Address, but we are often asked if the Email field, Address block, and Credit Card block can appear in a different order on the form.

I am assuming, as Eileen mentions in her blog (reference below) that it would be possible--after selecting a Payment Processor in the process of adding the Billing Widget--to be shown the fields required by that processor (or those processors) and for the processor to identify its required fields anywhere on the form by their associated tags.  That said, I am not actually familiar with tags, so it's very possible I am exaggerating their flexibility.

"Basically core needs to 'ask' the processor which fields to display rather than decide based on the processor's 'type' combined with it's payment type. I achieved some of this within the extension (in particular replacing BillingBlock.tpl with a block that has no hard-coded fields in it & which displays fields based on variables assigned to the template rather than template logic. However, I feel like I'm only starting to think through the metadata involved in 'instructing the form layer' and this is something I'm going to write more about - probably in a second blog."

A further note - in the iframe scenario there is an initial payment block (possibly empty - or with a credit card type field in the case of global collect) once this field has been filled out the iframe can be retrieved and rendered. Note that you would probably want an id to identify the contribution at this point.