CiviCRM Field Storage Engine for Drupal -- Proof of Concept

[Matt2000]

Hi all,

I have a very preliminary demo of using Drupal's Field Storage engine to expose CiviCRM data as Drupal fields, as discussed at CiviCon.

You can retrieve the code with git from my sandbox:

http://drupal.org/sandbox/matt2000/1512090

See the README for more info.

At the moment, it only supports Contact fields for the current Drupal user, but this is not inherent in the architecture, just a  shortcut to providing a quick demo.

I wanted to give other developers a chance to check things out early. I'll do a blog post once there are more interesting features to show off.

Please contact me if you want to help move this project forward with development or financial support.

==Proof-of-Concept demo==

1. Install CiviCRM and Webform CiviCRM module
2. Make sure your CiviCRM contact record conatins First Name, Last Name, and a Primary Email address
3. Enable civicrm_field_example
4. Visit 'node/add/civicrm-bio'
5. Note that your CiviCRM Contact data is pre-populated in the First Name, Last Name, and Email fields.
6. Add a title (required).
7. Save the node.
8. Note that CiviCRM data is displayed on the node view page.
9. Edit your Contact record in CiviCRM, changing one of the fields.
10. Refresh the node view; notice that the CiviCRM fields are automatically up-to-date.
11. Edit the node, and change one of the CiviCRM fields. Save.
12. In your CiviCRM Contact record, note that the change has been recorded.
13. Visit admin/structure/types/manage/civicrm_bio/display and admin/structure/types/manage/civicrm_bio/fields.
    Note that CiviCRM fields can be manipulated like other Drupal fields.

[Matt2000]

I got one question by email that I wanted answer here for the benefit of other interested developers:

>One question I haven't been clear on is how one could integrate business rules around CRUD operations between CiviCRM and Drupal using this approach. Are you calling the CiviCRM API for all of these operations, which in turn invokes the appropriate BAO functions?

Yes, every read/write operation passes through civicrm_api(), and I intend to strictly enforce that rule as development progress. By design, The idea is that Drupal can be completely ignorant of CiviCRMs internal representations of data. All Drupal knows is "this field data belongs to a CiviCRM object of type $t with ID $i".

So a developer can use either CiviCRM hooks or Drupal hooks to implement business rules, and it won't matter in the least. When the Drupal entity is saved, however that happens, it sends its data to Civi via the API, and when the Drupal entity is loaded however that happens, it reads it's data from CiviCRM via the API. (And we clear Drupal's internal field data cache when CiviCRM objects are updated in CiviCRM.)

[Coleman Watts]

This seems like a great foundation toward integration with software that is based on the field storage engine. Not sure if anything is out there yet that we want to integrate with, but this is certainly the way of the future for Drupal.

[Matt2000]

Not sure if anything is out there yet that we want to integrate with...

 

How about all of core -- node, user, taxonomy modules? How about being able to theme pages of civicrm data like any other drupal page? How about any of dozen of modules that use fields, like Display Suite, Field Group, Field Collection. Field Access, Auto Node Titles, Profile 2, various field widget modules, etc, etc...

A short step from here are things like EntityFieldQuery integration, Additional Views integration, etc, etc.

In short, this kind of module will be essential for allowing Drupal 7 application developers to use CiviCRM data without having to ignore their Drupal knowlegde and learn all of CiviCRM's alternate systems. They will be able  to just work with Entities, Fields, and themes, the same as everything else in Drupal. No more Smarty, PEAR::DB, Quickform, etc, etc. And it will be relatively easy to maintain, because it's only dependencies are the CiviCRM API v3 and Drupal Core.

Now we just have to build it... :-P 

[Coleman Watts]

Those are some good examples. Hopefully this means the next big drupal integration module won't be starting from scratch (unless you count this one).
I guess I'm still trying to wrap my head around this... it's nice that you can push/pull civicrm entities in drupal, but the question is, which one? Your POC only deals with the active user, but figuring out what contact (or more subtly, which of their phone numbers) we are talking about is a real trick -- at least it's proven very tricky with the webform stuff.
Is that something this framework will address, or will it be up to the module that uses it to figure out?

[Matt2000]

To the degree there is a correct answer to that challenge, and you've discovered it for Webform CiviCRM, I was hoping to be able to reuse your work there. :-)

Beyond that, for dealing with other contacts, I was planning to leverage the existing CiviCRM Contact Reference field to designate when a particular entity or field group should reference some other contact. And/or we could make that a configuration option per field. (i.e., When you configure field B, you select some other field A which stores a reference to the appropriate entity & id for the data in field B.)

And beyond that, it's up to the application, in the same way as it is using the API now.

In case I didn't mention it before, although the POC has no GUI for creating fields like other fields are created now, we certainly could provide one. We could extend the Core Field UI module, but I suspect it'd be even better to adapt the Webform CiviCRM GUI, since it's already "aware" of some of the issues you're mentioning.

[Andrew Perry]

Thanks for both of your work on improving the integration between Drupal and CiviCRM - your work does and will make life so much easier when it comes to tailoring the user's experience when contributing and consuming CiviCRM data!

[xavier]

Hi,

don't know if it is of any use, but there are getvalue and setvalue actions (the later as of 4.2).

We don't cache anything, to if you modify two fields of the same entity, that's going to be two sql update, so use with care

[joemaine]

Hey Matt,

...do you understand what I'm trying to do with CiviCRM and Drupal.. http://drupal.org/node/1618108 

[Matt2000]

Yes, that's exactly what my sandbox project is demonstrating -- attaching CiviCRM data to any fieldable Drupal entity.

There's no UI for it yet,  and there probably won't be until an organization can sponsor the development work. So you can grab my sandbox code and extend it if you're comfortable with Drupal field API code and the CiviCRM API, or you use the contactreference field + views approach that peter mentioned, but of course, that only works for Contact data, and it's "read-only" from the node.

[Coleman Watts]

I think I'm starting to catch on...
I just thought of a use-case for this -- people have been using various workarounds to sync Civi events to Drupal nodes so that they can be displayed in a calendar.
With this Field API solution, it wouldn't be a matter of syncing... the calendar would actually be pulling the data from Civi transparently through the node.

[Matt2000]

Sure, that's one of many things you could do.

Another way to think of this project is as a Drupal-centric replacement for CiviCRM Profiles -- Grab any group of CiviCRM fields and put them into a user-facing form (node form, user form, profile2 form, whatever), and enhance them with Drupal modules, style them with Drupal theming, query them with Drupal API functions, control access with Drupal permissions, etc, etc....

[joemaine]

Matt - if you could pull this together...a whole new world for CiviCRM opens up.

[sonicthoughts]

Hi Matt - This would be incredibly valuable.  Just getting started with display suite and it could totally transform development integration w/ drupal if fields were exposed.  The hooks are already there although it could probably be done today with w/ php custom fields,(I'm sure there is a much more elegant way.)  I love the ability of DS to customize anything. Any progress on this from your end?

[Matt2000]

No progress. My most recent Civi Project ended not needing much Drupal integration, and no other sponsors have come forward, and I'm starting a new full-time contract soon. As far as I can tell, not enough people have seen the value of this project, at least not people who have come forward with skills to volunteer  or financial sponsorship. So I don't think this will happen. I might consider working on it on the occasional weekend if there were financial support, but my existing clients are using Civi less and less, so I'm probably not going to self-fund this, and I certainly can't commit to providing technical support and on-going maintenance myself. So this is probably dead, unfortunately.