Implementing a custom ACL system in CiviCRM

[Chris Burgess]

Our (Drupal5+CiviCRM2.0) site needed a regional ACL implementation in order to give us the ability to easily grant access for our internal staff to contacts in their geographic region. Because we have staff at a provincial, regional and branch level, we needed to be able to configure permissions for each level.

We did this by implementing code which overrides the ACL implementation in CiviCRM. I'd appreciate any input and suggestions on our methodology - it's a work in progress, but it's working well for us so far.

I haven't tested this with Joomla, but I hope that my notes below help someone roll a similar implementation there.

Running custom local code

First of all, I wanted to be able to make local code changes and still easily track CiviCRM SVN updates to v2.0 branch. At the CiviCRM NZ meetup, Lobo worked with us to create a method (available in 2.1 by default) to override CiviCRM code by inserting your own into the include path before CiviCRM's default files get loaded (just like the template system does).

You can skip this step if your normal method is just to hack the local source ... that's how I used to work too - but I'm getting too lazy to keep track of local modifications

So: to make this happen on 2.0, you need to (short form):

Code: [Select]

cd $CIVICRM_PATH
svn merge -r14315:14316 http://svn.civicrm.org/civicrm/trunk .
svn merge -r14329:14330 http://svn.civicrm.org/civicrm/trunk .

Once this is done, there's a new setting in CiviCRM › Administer CiviCRM › Global Settings › Directories labelled Custom PHP Path Directory. Then we need to create a new CRM directory, and copy any .php file we wish to modify to that folder (preserving the original CRM/Foo/Bar/Baz.php path).

Now we're ready to build in our own ACLs!

Setting up the custom data

Each contact in our system has a custom data group named "Geography", and in this group they will have a custom data field for each of "Province", "Electorate" and "Branch". The simplest method would have been to give our staff access only to people within their OWN province, electorate or branch, but this didn't fit our organisational model - so we needed to add a second custom data group named "Regional Access" and insert a multi-select here for each province, electorate and branch, allowing a staff user to have access to any number of regional areas.

We also wanted to add a second ACL level. We have some "VIP" contacts, who we don't want ordinary staff contacting. We added a VIP contact custom data field, and we grant access to these VIP contacts ONLY to users who have a custom permission we define, "headoffice view".

Implementing a local ACL

Now it's time to add our local ACL rules, which will override the normal ACL rules.

These rules will only apply if a user does NOT have "Edit all contacts" permission granted via their Drupal role. That permission will see that the ACL checks through the system are skipped completely.

To keep things brief, here's the original ACL.php file we modify. We replace a block of the function CRM_ACL_BAO_ACL::whereClause() to inject our own query logic which checks drupal permissions and custom data for the logged-in user against the custom data of the contact that is (potentially) being accessed.

Code: [Select]

    public static function whereClause( $type, &$tables, &$whereTables, $contactID = null ) {
        require_once 'CRM/ACL/BAO/Cache.php';

        $acls =& CRM_ACL_BAO_Cache::build( $contactID );
        // CRM_Core_Error::debug( "a: $contactID", $acls );

$ids = array( );
        if ( ! empty( $acls ) ) {
  $aclKeys = array_keys( $acls );
  $aclKeys = implode( ',', $aclKeys );
 
  $query = "
SELECT   a.operation, a.object_id
  FROM   civicrm_acl_cache c, civicrm_acl a
 WHERE   c.acl_id       =  a.id
   AND   a.is_active    =  1
   AND   a.object_table = 'civicrm_saved_search'
   AND   a.id        IN ( $aclKeys )
ORDER BY a.object_id
";
  //CRM_Core_Error::debug( 'q', $query );

  $dao =& CRM_Core_DAO::executeQuery( $query, CRM_Core_DAO::$_nullArray );
 
  // do an or of all the where clauses u see
  while ( $dao->fetch( ) ) {
            if ( ! $dao->object_id ) {
      return ' ( 1 ) ';
            }

            // make sure operation matches the type TODO
            if ( $type == CRM_ACL_API::VIEW ||
                 ( $type == CRM_ACL_API::EDIT &&
                   $dao->operation == 'Edit' || $dao->operation == 'All' ) ) {
      $ids[] = $dao->object_id;
            }
  }
        }

$clauses = array( );
        if ( ! empty( $ids ) ) {
  $ids = implode( ',', $ids );
  $query = "
SELECT g.where_clause, g.select_tables, g.where_tables
  FROM civicrm_group g
 WHERE g.id IN ( $ids )
";
  $dao =& CRM_Core_DAO::executeQuery( $query, CRM_Core_DAO::$_nullArray );
  while ( $dao->fetch( ) ) {
            // currently operation is restrcited to VIEW/EDIT
            if ( $dao->where_clause ) {
      $clauses[] = $dao->where_clause;
      if ( $dao->select_tables ) {
$tables = array_merge( $tables,
       unserialize( $dao->select_tables ) );
      }
      if ( $dao->where_tables ) {
$whereTables = array_merge( $whereTables,
    unserialize( $dao->where_tables ) );
      }
            }
  }
}

/**
* modifications:
* 1. Exclude VIPs unless we have headoffice view
* 2. Restrict to logged in user's granted province/branch/electorate
*/
if ( !user_access( 'headoffice view' ) ) {
  /**
   * Exclude access to users with vip_status = 1
   */
  $denyClauses[] = 'custom_value_1_Special_Data.vip_status != 1 OR custom_value_1_Special_Data.vip_status IS NULL' ;
  $whereTables['custom_value_1_Special_Data'] =
    $tables['custom_value_1_Special_Data'] =
    'LEFT JOIN custom_value_1_Special_Data ON custom_value_1_Special_Data.entity_id = contact_a.id' ;
  /**
   * Restrict access to contacts whose electorate matches the
   * current user's regional_access.electorates
   *
   * Load custom data with regional ACLs for this logged in
   * user, and construct extra whereClauses to ensure they
   * don't see outside their sandbox
   */
         /*
          * Get the logged-in user's contact ID
          */
  $session   =& CRM_Core_Session::singleton( );
  $contactID =  $session->get( 'userID' );
  /**
   * TODO: optimise these three queries
   */
         /*
          * Find the custom field IDs of the fields we restrict on, based on
          * the field label and custom data group title
          */
  $electorate_field_id = CRM_Core_BAO_CustomField::getCustomFieldId( array( 'field_label' => 'Electorates',
    'group_title' => 'Regional Access', ) ) ;
  $province_field_id   = CRM_Core_BAO_CustomField::getCustomFieldId( array( 'field_label' => 'Provinces',
    'group_title' => 'Regional Access', ) ) ;
  $branch_field_id     = CRM_Core_BAO_CustomField::getCustomFieldId( array( 'field_label' => 'Branches',
    'group_title' => 'Regional Access', ) ) ;
  $electorate_field_name = 'custom_' . $electorate_field_id ;
  $province_field_name   = 'custom_' . $province_field_id ;
  $branch_field_name     = 'custom_' . $branch_field_id ;
  $params    = array( 'contact_id' => $contactID,
      'return.'.$electorate_field_name => 1,
      'return.'.$province_field_name => 1,
      'return.'.$branch_field_name => 1
      ) ;
  require_once('api/v2/Contact.php');
  $contact   = civicrm_contact_get( $params ) ;
  // add electorate ACL
  if ( isset( $contact[$electorate_field_name] ) ) {
    $electorates = explode( CRM_Core_DAO::VALUE_SEPARATOR, $contact[$electorate_field_name] ) ;
    // remove empty strings which appear at beginning and end of array
    $electorates = array_diff( $electorates, array( '' ) ) ;
    if ( !empty( $electorates ) ) {
      $sqlElectorates = "'" . implode("','", $electorates) . "'" ;
      $clauses[] = 'custom_value_1_Geography.electorate IN ( ' . $sqlElectorates . ' )' ;
      $whereTables['custom_value_1_Geography'] = $tables['custom_value_1_Geography'] =
'LEFT JOIN custom_value_1_Geography ON custom_value_1_Geography.entity_id = contact_a.id' ;
    }
    //     $clauses[] = 'custom_value_1_Geography.electorate LIKE \'%\'' ;
  }
  // add province ACL
  if ( isset( $contact[$province_field_name] ) ) {
    $provinces = explode( CRM_Core_DAO::VALUE_SEPARATOR, $contact[$province_field_name] ) ;
    //     CRM_Core_Error::debug($provinces);
    // remove empty strings which appear at beginning and end of array
    $provinces = array_diff( $provinces, array( '' ) ) ;
    if ( !empty( $provinces ) ) {
      $sqlProvinces = "'" . implode("','", $provinces) . "'" ;
      $clauses[] = 'custom_value_1_Geography.province IN ( ' . $sqlProvinces . ' )' ;
      $whereTables['custom_value_1_Geography'] = $tables['custom_value_1_Geography'] =
'LEFT JOIN custom_value_1_Geography ON custom_value_1_Geography.entity_id = contact_a.id' ;
    }
  }
  // add branch ACL
  if ( isset( $contact[$branch_field_name] ) ) {
    $branchs = explode( CRM_Core_DAO::VALUE_SEPARATOR, $contact[$branch_field_name] ) ;
    // remove empty strings which appear at beginning and end of array
    $branchs = array_diff( $branchs, array( '' ) ) ;
    if ( !empty( $branchs ) ) {
      $sqlBranchs = "'" . implode("','", $branchs) . "'" ;
      $clauses[] = 'custom_value_1_Geography.branch IN ( ' . $sqlBranchs . ' )' ;
      $whereTables['custom_value_1_Geography'] = $tables['custom_value_1_Geography'] =
'LEFT JOIN custom_value_1_Geography ON custom_value_1_Geography.entity_id = contact_a.id' ;
    }
  }
}
$dbc = array( $clauses, $whereTables );
// CRM_Core_Error::debug_var('qry', $dbc);
        /**
         * In the original function, all the various clauses are OR'd together here.
         *
         * We want to ensure that one of our clauses - the VIP exclusion - is AND'd,
         * so we have kept it separate from the other $clauses, in $denyClauses
         */
if ( ! empty( $clauses ) ) {
  $restr = ' ( ' . implode( ' OR ', $clauses ) . ' ) ' ;
  if ( ! empty( $denyClauses ) ) {
    $restr .= ' AND ( ' . implode( ' AND ', $denyClauses ) . ' )' ;
  }
        } else {
           $restr = ' ( 0 ) ';
        }
// CRM_Core_Error::debug_var('restr', $restr);
return $restr ;
    }

Then, because we need to be able to efficiently look up the custom field ID in order to do the above, we also copy into our custom PHP CRM/Core/BAO/CustomField.php and add this additional function.

Code: [Select]

    /**
     * Make it possible to find a generated custom field ID by looking
     * up the field's label / column name
     *
     * @param array details to identify the custom field - ideally,
     * custom group name as well as either custom field label or name,
     * but custom group name / title is not required
     *
     * @return custom field ID
     */
    public static function getCustomFieldId( $params ) {
      if ( !is_array( $params ) ||
   ( !isset( $params['field_name'] ) &&
     !isset( $params['field_label'] ) ) ) {
return civicrm_create_error( "Custom field name or label is required" ) ;
      }
      $args = $where = array() ;
      if ( isset( $params['group_title'] ) || isset( $params['group_name'] ) ) {
$join = "JOIN civicrm_custom_group ON civicrm_custom_group.id = civicrm_custom_field.custom_group_id" ;
if ( isset( $params['group_title'] ) ) {
  $where[] = "civicrm_custom_group.title = %1" ;
  $args[1] = array( $params['group_title'], 'String' ) ;
}
if ( isset( $params['group_name'] ) ) {
  $where[] = "civicrm_custom_group.name = %2" ;
  $args[2] = array( $params['group_name'], 'String' ) ;
}
      }
      if ( isset( $params['field_name'] ) ) {
$where[] = "civicrm_custom_field.column_name = %3" ;
$args[3] = array( $params['field_name'], 'String' )  ;
      }
      if ( isset( $params['field_label'] ) ) {
$where[] = "civicrm_custom_field.column_name = %4" ;
$args[4] = array( $params['field_label'], 'String' ) ;
      }
      $sql = "SELECT civicrm_custom_field.id FROM civicrm_custom_field " .
$join .
" WHERE " . implode( " AND ", $where ) ;
      return CRM_Core_DAO::singleValueQuery( $sql, $args );
    }

(I'm hoping similar functionality to my CRM_Core_BAO_CustomField::getCustomFieldID() will be in 2.1 by default - or that someone will show me the existing function I missed!)

I'm sure there's plenty of room for improvement, but that's what we started with.

The results

As intended, the users who don't have "edit all users" (or "view all users") permission see no people, until they are assigned some regional access entries under that custom data group. If your implementation wanted to recycle your geography - or even non-custom data like the "State" or "City" field - it could do exactly that, and even simpler.

Things to investigate further

Can this be made more efficient? Can some of this data be cached? I'm pretty sure the above code is evaluated only once per query (eg for a contact search) applying its custom SQL restriction to the query (and thus the resultset).

What unexpected results come of this? (I'll follow this up in a moment! We hit one already.)

Your thoughts / input / criticism welcome - please dig in

[Donald Lobo]

Chris:

thanx for the incredibly detailed report. I've cross posted this article on the blog here: http://civicrm.org/node/381. For other readers, a related blog entry is Another approach to ACL and permissioning for hierarchical organizations

lobos

[Chris Burgess]

Cheers Lobo. Thanks again for your input and thought today, too!

So ... the ACLs worked really well, but after a short while some of our newly ACL'd users reported oddities when working with CiviCRM groups. In some contexts, groups appeared to work OK, but in other places they either didn't display at all (for the most part) or behaved somewhat oddly (peterd reported that he selected and added three members to a group, but then was told that he'd added 22,000 members to the group successfully ... I didn't manage to duplicate this one though, maybe pete can fill us in).

I talked to Lobo, who explained that the groups weren't appearing because CiviCRM was detecting that the current user didn't have access to any given group's complete member set. He suggested that we think about ACL'ing our groups to ensure that people's access to a group was only granted when they had access to its complete member set anyway - this could avoid potential confusion eg with CiviMail (where the mail job is sent out by cron using a system account anyway, and might not enforce the user restrictions applied when the mail job was set up) or where a member from province A and a member from province B are discussing the virtual group "subscribers", which actually is a different set of people based on geography in our case.

Our organisational model uses geography as a separate dimension from the groups we refer to - so this wasn't such a concern for us as the fact that no groups were available to our ACL'd users. The simple fix would be to grant all ACL'd users "edit all contacts" permission, but that would subvert the ACL entirely, so it would be a backward step.

The brutish, ugly and hopefully temporary fix was to shortcircuit the ACL controls applied to groups, so that's what I did today.

The places where groups weren't visible were the manage groups page, and the contact's groups tab.

First, I had a crack at implementing an override using an additional Drupal permission check for the CiviCRM permission "edit groups", by implementing the CRM_ACL_BAO_ACL::group() function in our own custom ACL.php:

Code: [Select]

    /**
     * ACL filter the groups which the current user has access to
     *
     *
     */
    public static function group( $type,
                                  $contactID = null,
                                  $tableName = 'civicrm_saved_search',
                                  $allGroups = null ) {
        require_once 'CRM/ACL/BAO/Cache.php';

        $acls =& CRM_ACL_BAO_Cache::build( $contactID );

        $ids  = array( );
        if ( empty( $acls ) ) {
            return $ids;
        }
       
// $args = func_get_args() ;
// CRM_Core_Error::debug( 'args', $args ) ;

/**
* CiviCRM permission "edit groups" can override the builtin group ACLs
*/
if ( CRM_Core_Permission::check( 'edit groups' ) && $tableName == 'civicrm_saved_search' ) {
  if ( !is_null( $allGroups ) ) {
    return array_keys( $allGroups ) ;
  }
}

        $aclKeys = array_keys( $acls );
        $aclKeys = implode( ',', $aclKeys );

        $query = "
SELECT   a.operation, a.object_id
  FROM   civicrm_acl_cache c, civicrm_acl a
 WHERE   c.acl_id       =  a.id
   AND   a.is_active    =  1
   AND   a.object_table = %1
   AND   a.id        IN ( $aclKeys )
ORDER BY a.object_id
";
        $params = array( 1 => array( $tableName, 'String' ) );

        // CRM_Core_Error::debug( $query, $params );

        $dao =& CRM_Core_DAO::executeQuery( $query, $params );

        while ( $dao->fetch( ) ) {
            if ( $dao->object_id ) {
                if ( $type == CRM_ACL_API::VIEW ||
                     ( $type == CRM_ACL_API::EDIT &&
                       $dao->operation == 'Edit' || $dao->operation == 'All' ) ) {
                    $ids[] = $dao->object_id;
                }
            } else {
                // this user has got the permission for all objects of this type
                if ( $type == CRM_ACL_API::VIEW ||
                     ( $type == CRM_ACL_API::EDIT &&
                       $dao->operation == 'Edit' || $dao->operation == 'All' ) ) {
                    if ( ! empty( $allGroups ) ) {
                        return array_keys( $allGroups );
                    } else {
                        return array( );
                    }
                }
            }
        }
       
        return $ids;
    }

But that didn't do what I expected - and when I traced the groups which were getting passed INTO this function, I found they often had already been removed by something further up the tree.

I'll spare the detail - it's not so interesting when it only leads to an ugly hack - and show only the changes I made. Again, I copied these files (CRM/ACL/API.php, CRM/Contact/BAO/GroupContact.php, and CRM/Core/PseudoConstant.php (see note below)) into my custom code path and applied these changes in the separate directory.

Code: [Select]

Index: ACL/API.php
===================================================================
--- ACL/API.php (revision 14412)
+++ ACL/API.php (working copy)
@@ -136,6 +136,8 @@
     public static function groupPermission( $type, $groupID, $contactID = null, $tableName = 'civicrm_saved_search' ) {
         static $cache = array( );
 
+/* cmb - hack to ignore group perms */
+return true ;
         $key = "{$tableName}_{$type}_{$contactID}";
         if ( array_key_exists( $key, $cache ) ) {
             $groups =& $cache[$key];
Index: Contact/BAO/GroupContact.php
===================================================================
--- Contact/BAO/GroupContact.php (revision 14412)
+++ Contact/BAO/GroupContact.php (working copy)
@@ -349,12 +349,12 @@
                              'civicrm_group'                => 1,
                              'civicrm_subscription_history' => 1 );
         $whereTables = array( );
-        if ( $ignorePermission ) {
+        if ( $ignorePermission || true /* cmb - hack to ignore group perms */ ) {
             $permission = ' ( 1 ) ';
         } else {
             $permission = CRM_Core_Permission::whereClause( CRM_Core_Permission::VIEW, $tables, $whereTables );
         }
-       
+
         require_once 'CRM/Contact/BAO/Query.php';
         $from = CRM_Contact_BAO_Query::fromClause( $tables );

... I did say it was ugly

PS. I removed the changes to CRM/Core/PseudoConstant.php from our live site and this patch entry - take note that some CiviCRM files are not able to be overridden using the custom path method above, as they are already loaded when CiviCRM adds them to the load path; if a later require() or include() references those files in the custom location, PHP will exit with "Cannot redeclare class CRM_.... in $custom_dir". The PseudoConstant modification didn't work for me initially, and once I'd completed these other changes it wasn't required.

[petednz]

Just to clarify the 'odd' behaviour. When logged in as a user who has 'regional access' limitations (and therefore no access to 'edit all contacts) I tried creating a group (with 3 contacts). System allowed the group to be created but the user could not see it. When I checked the status of the group using my Admin login I found it had added everyone contact in the database.

[lcdweb]

VERY cool that there will be a php override directory in 2.1 -- I also struggle to maintain modified files through upgrades. That will be a huge help.

[Donald Lobo]

Note that i've added two acl hooks (aclWhereClause and aclGroup) to the acl code. In 2.1, you should be able to do the above stuff in a custom module hook rather than hacking civicrm core / overwriting files

lobo
 

[matth3wh]

Just trying to clarify a few things...  the ability to put in Custom Templates and Custom PHP Path has been included... since civicrm 2.1
I take a copy of the original ACL.php / etc and put it in to the Custom PHP directory and then modify that.

So using acl hooks (aclWhereClause and aclGroup) I could in theory limit which Activity Types can been seen by a particular group of users.
e.g.  Team MOB  should only see Activity type - MOB Follow Up
   and   Team MEM  cannot see Activity type - MOB Follow Up

?

[Donald Lobo]

hooks basically are called by the system to "extend" the system. You should not make a copy of the core code files . You can read up on hook on drupal.org (which basically is where we got them from)

currently we permissions: contacts, groups, custom groups and events. So you can extend permissioning on those objects. Activities are not permissioned directly, but are permissioned indirectly via contacts, i.e. u should be able to see activities that involve contacts that you are permissioned for.

lobo

[matth3wh]

Oops, I thought in the OP's post they were saying they hacked ACL.php and CustomField.php ?  

[edit] SNIP - Oops just went off topic... jogged by thinking about possible ACL Hooks in to Activities

[Donald Lobo]

yes, hence we introduced the hook to avoid the hacks

i'm not sure i understand completely the rest of the message, so skipping it for now. does not seem related to acl hooks anyway

you can extend civicrm a fair amount and add functionality as you see fit

lobo