CiviCRM Community Forums (archive)

(Note: In true saloon fashion, I'm posting a proposal that should merit a dozen -1's. That's OK. Even if it goes nowhere, I think this write-up is useful in examining why certain issues exist and persist. And I want to have a document to reference whenever people ask about why Civi becomes a big busy-body when you toggle a Drupal module/Joomla extension/WordPress plugin.)

A standard matter in developing frameworks and large applications is providing the glue to connect together different modules. This glue almost always addresses two problems:

• Events - One module fires an event, and other modules respond to it. The general goal is to provide an open-ended message-passing mechanism. This feature has a few different variations ("Events", "Observers", "Listeners", "Publish/subscribe", "Drupal Hooks", "WordPress Actions+Filters", "Joomla Plugins", etc).
• Declarations - One module declares a list of things (e.g. URL routes, menu entries, database tables, blocks, classes, activity-types, case-types, custom-fields). Another module processes this list, ensuring that those things are created.

Events and declarations are sometimes (but not always) linked. For example, consider the problem of declaring permissions. In Joomla, a developer declares permissions in a purpose-built XML file. In Drupal, however, a Drupal module listens to the hook_permissions event and returns a list. There are, of course, trade-offs in the two approaches (e.g. for a developer browsing the file system, it's easier to drilldown to Joomla's XML file; for a developer creating a dynamic/user-configurable module, the hook is more flexible). However, both those are widely used platforms that easily handle the "80%" use-cases. Each has been polished and supplemented to the point that it makes a complete system.

In Civi, we sit at the intersection of several different communities (Drupal, Joomla, WordPress; developer, implementer, user), and this has affected our approach to events and declarations. If you don't mind putting on the Indiana Jones hat, we can dig into the exciting, rabble-rousing world of software-archaeology for a moment:

• For events, Civi avoided defining its own event system. Civi 1.x had a "component" system, but it was decidedly KISS and used adhoc eventing to pass messages between components. After 7 years, Civi 4.1 added hooks for module-extensions. (But this only works with module-extensions -- not with "components".) For the most part, we sought to externalize the events -- i.e. we wrote adapters to use the Drupal, Joomla, (and now, in 4.6) WordPress event systems.
• For declarations, you find examples by inspired anything/everything. hook_xmlMenu (from Civi 2.x?) is Drupalish (with a hook) and *also* Joomlaish (with separate files). hook_navigationMenu (from Civi 3.x) is decidedly Drupalish, and the payment-extensions (from Civi 3.x) are decidedly Joomlaish. (Cue Harrison Ford again... he's in a bar on Tatooine, and the funky blue hippo plays keyboard while implementing hook_xmlMenu. It's a colorful, diverse world.)

This approach has had a logic to it -- engaging developers who grew up on other platforms (Drupal, Joomla, WordPress). The theory: if you grew up using hooks in Drupal, then you can transition easily to hooks in Civi. If you grew up using XML in Joomla extensions, then you can transition easily to XML in Civi extensions. I suspect that half the people reading this came in through that engagement strategy. To some extent, this has been succesful and tactically brilliant. Never-the-less, I'm going to state the case for an alternative. If you came in this way, you should downvote my alternative.

Proposal: Civi should have one native pattern for events and declarations which works the same way in all package types (i.e. in Civi core, in Civi extensions, in Drupal modules, in Joomla extensions, and in WordPress plugins). Third-party addons (pregardless of packaging format) should communicate with Civi through the Civi event system - not through the CMS event system.

Why:

• The current approach based on event-adapters requires that we cater to the lowest-common denominator. For example, when activating any Drupal/Joomla/WordPress/Civi module, Civi 4.x triggers a full system flush on the off-chance that the new module declares something using one of our hooks. This wouldn't normally be necessary for Drupal modules (because you can examine/invoke hooks on a fine-grained, per-module basis), but that technique isn't valid on Joomla. It wouldn't normally be necessary in Joomla (because declarations are handled by the XML files, which are broken down per-extension), but Drupal doesn't have a separate declaration convention - it only has hooks. So we're left collecting declarations with the only technique that works everywhere: fire a whole bunch of hooks at once.
• A single system allows us to develop a cogent feature-set. It's not that we want or need all the features of every CMS combined into one uber-system. Rather, we need a set of features that form a consistent whole - and the lowest-common denominator is not a consistent whole.
• A single system requires less maintenance -- we don't need to refactor/bugfix on D6/D7/D8/J/WP.
• A single system allows consistent documentation -- we don't need to describe 3-4 notations for doing the same thing.
• A single system allows more shared tooling. For example, civix today only works with native extensions, but most of its output could work verbatim in CMS packages. The obstacle is that each platform requires events/declarations be in different notation.
• A single system can be used by core code (e.g. by components), which allows us to make looser coupling between the core components.
• You might expect that pushing Civi events through the CMS event system enables deep integrations (like civicrm/drupal-views or civicrm/drupal-rules). This is not true. Deep-integration comes from using Civi interfaces and CMS interfaces in the same package. It's perfectly functional to write the CMS-bits using CMS-conventions and write the Civi-bits using Civi-conventions - as long as they're packaged together in the same deliverable.
• The engagement benefit of mixing notations in pidgin fashion is questionable. For every Drupal dev who got lured in by a Drupalism, there's another Drupal dev who got scared off by some Joomlaism, and there's a WordPress dev who couldn't figure out how to adapt a Drupal-Civi tutorial to his platform, and there's an unaligned dev who got all judgmental about the inconsistencies.

Design: Not sure yet. I like http://symfony.com/doc/current/components/event_dispatcher/index.html and http://symfony.com/doc/current/components/dependency_injection/introduction.html as building blocks. We've made some babysteps in 4.5/4.6, but there's still some initialization issues (i.e. when/how does each CMS package get a reference to Civi's event dispatcher).

How: Not sure. Some ideas:

• Create a transition period where, e.g., v5.0 supports all the existing hooks on legacy event systems -- plus adds a new system. Mark all the CMS event systems as deprecated; don't expose new hooks on them; remove in 5.1 or 6.0.
• Make a clean break in a major version update (e.g. 5.0) - kill the current hook system. But that breaks...basically... all of contrib.
• As above, but patch-welcome contributions to the civicrm-drupal, civicrm-joomla, and civicrm-wordpress integrations to provide adapters between the old and new event systems. Some CMS's may get adapters; some may not; depends on who steps forward.

I have to admit that I was lured to CiviCRM programming through the hook system, but then it made it  difficult to understand the logic of core itself. Not to mention that the "yay this is similar to Drupal" pretty quickly turns to bitterness when one realizes that QuickForm is not exactly Drupal's FormAPI.

That seems to be a tough transition. I'm glad you survived it, though.

If I understand correctly, you're proposing to have a single way to declare both "menu routes" URL declarations (currently in the XML, joomla-ish) and hook callbacks for events (drupal-ish)? (to simplify)

Yup, so just to pull an example out of thin air, we might say that all callback functions use a @Civi\Event annotation, and all declarations (menu routes, navbar items, etc) will use an array-alter pattern. In any package (Drupal/Joomla/WordPress/Civi based), you can drop-in a *.php file with suitable annotations. For example:

<?php
// FILE: sites/all/modules/mymodule/Foo.php
namespace mymodule;
class Foo extends CRM_Core_Page {
  /**
   * @Civi\Event("civicrm.routes")
   */
  public function routes(&$routes) {
    $routes['some/path'] = array(
      'callback' => 'run',
    );
  }
  public function run() {
    echo "Hello world";
  }
}

If we wanted something that skewed closer to traditional Drupal, we could say that -- in any package -- the developer may drop in "*.civi.php" files. These files contain namespaced functions:

<?php
// FILE: sites/all/modules/mymodule/mymodule.civi.php
namespace mymodule;
function hook_civicrm_routes(&$routes) {
    $routes['some/path'] = array(
      'callback' => 'CRM_Mymodule_Page_Mypage',
    );
}
// FILE: sites/all/modules/mymodule/CRM/Mymodule/Page/Mypage.php
class CRM_Mymodule_Page_Mypage extends CRM_Core_Page {
  public function run() {
    echo "Hello world";
  }
}

Those are purely examples -- IMHO, these details are secondary to the thesis that we should have a single notation supported in all package types.

A couple points which are tangential but which also touch on the idea of re-thinking the event subsystem:

1) Webhooks. Find a way to tap into the spigot of Civi events/hooks and pass them to external processes.

2) In the long-term, as the PHP community continues to do a better job of reusing libraries, we may find that the current architecture boxes us in vis-a-vis library compatibility. (For example, future versions of Civi and Drupal might both use Guzzle; but if they have diff versions of Guzzle, then there could be conflicts.) This arises because of how we've linked CMS+CRM in the same process-space - which we've done to facilitate events/message-passing. However, if we change the pattern for events/message-passing, then maybe we can get the same deep integration while running in separate processes. It's not obvious how to do that while maintaining the current performance profile and deployment process, but that might be interesting to think on...