CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site
This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • APIs and Hooks (Moderator: Donald Lobo) »
  • Performance & the API (& caching that isn't)
Pages: [1]

Author Topic: Performance & the API (& caching that isn't)  (Read 804 times)

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Performance & the API (& caching that isn't)
July 22, 2011, 01:22:39 am
Hi,

I have been finding the activity API slower than the rest & having a script that creates a couple of thousand activities I decided to do some tests.

258.77 secsTotal time taken by script 
   (35.31 secs - Script overhead (no activity create ) )
228.37 secs  - Script time without API contact checks
210.60 secs - Script time without API checks & DAO checks
141.53 secs - Script time without API checks & calls to PseudoConstant::activityType

So, in summary
106 secs = API time without the 3 checks
223 secs = API time with the 3 checks.

A little more than half the time is spent checking & half of that is in calls to a PSEUDOCONSTANT that is obviously not working right. (ie. it seems to run the same query over & over although I can't see any obvious reason)

Here are the 3 checks:

1)         
Code: [Select]
$sql = '
SELECT  count(*)
  FROM  civicrm_contact
 WHERE  id IN (' . implode( ', ', $valueIds ) . ' )';
        if ( count( $valueIds ) !=  CRM_Core_DAO::singleValueQuery( $sql ) ) {
            return civicrm_api3_create_error( 'Invalid '. ucfirst($key) .' Contact Id' );
        }
2)
Code: [Select]
    foreach ( $activityIds as $id => $value ) {
        if (  $value &&
              !CRM_Core_DAO::getFieldValue( 'CRM_Activity_DAO_Activity', $value, 'id' ) ) {
            return civicrm_api3_create_error(  'Invalid ' . ucfirst( $id ) . ' Id' );
        }
    }
3)
Code: [Select]
    require_once 'CRM/Core/PseudoConstant.php';
    $activityTypes = CRM_Core_PseudoConstant::activityType( true, true, true, 'name', true );

    $activityName   = CRM_Utils_Array::value( 'activity_name', $params );
    $activityTypeId = CRM_Utils_Array::value( 'activity_type_id', $params );

    if ( $activityName ) {
        $activityNameId = array_search( ucfirst( $activityName ), $activityTypes );

        if ( !$activityNameId ) {
            return civicrm_api3_create_error(  'Invalid Activity Name'  );
        } else if ( $activityTypeId && ( $activityTypeId != $activityNameId ) ) {
            return civicrm_api3_create_error(  'Mismatch in Activity'  );
        }
        $params['activity_type_id'] = $activityNameId;
    } else if ( $activityTypeId &&
                !array_key_exists( $activityTypeId, $activityTypes ) ) {
        return civicrm_api3_create_error( 'Invalid Activity Type ID' );
    }


Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Re: Performance & the API (& caching that isn't)
July 22, 2011, 01:28:40 am
& here's the backtrace on that pseudoconstant. (these are actually 2 separate calls from back-to-back api calls - the timestamp seems to be from when the script started running)

Code: [Select]
Jul 22 20:26:25  [info] $
SELECT  v.name as name ,v.value as value, v.grouping as grouping
FROM   civicrm_option_value v,
       civicrm_option_group g
WHERE  v.option_group_id = g.id
  AND  g.name            = 'activity_type'
  AND  g.is_active       = 1  AND  v.is_active = 1  AND  ( v.component_id IS NULL OR v.component_id IN (1,2,3,4,6,7,8,9))  ORDER BY v.weight = C:\utils\eclipseworkspace\civicrm-3.4\packages\DB\DataObject.php, backtrace, 2358
C:\utils\eclipseworkspace\civicrm-3.4\packages\DB\DataObject.php, _query, 1610
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\DAO.php, query, 147
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\DAO.php, query, 874
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\OptionGroup.php, executeQuery, 109
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\PseudoConstant.php, values, 503
C:\utils\eclipseworkspace\civicrm-3.4\api\v3\Activity.php, activityType, 296
C:\utils\eclipseworkspace\civicrm-3.4\api\v3\Activity.php, _civicrm_api3_activity_check_params, 82
C:\utils\eclipseworkspace\civicrm-3.4\api\api.php, civicrm_api3_activity_create, 134
C:\utils\eclipseworkspace\wcan\drupal\modules\surveyactions\surveyactions.drush.inc, civicrm_api, 244
C:\utils\eclipseworkspace\wcan\drupal\modules\surveyactions\surveyactions.drush.inc, createActivitiesForGroup, 154
, surveyactions_fill_groups,
C:\utils\drush\includes\drush.inc, call_user_func_array, 51
C:\utils\drush\drush.php, drush_dispatch, 90
C:\utils\drush\drush.php, drush_main, 40

Code: [Select]
Jul 22 20:26:25  [info] $
SELECT  v.name as name ,v.value as value, v.grouping as grouping
FROM   civicrm_option_value v,
       civicrm_option_group g
WHERE  v.option_group_id = g.id
  AND  g.name            = 'activity_type'
  AND  g.is_active       = 1  AND  v.is_active = 1  AND  ( v.component_id IS NULL OR v.component_id IN (1,2,3,4,6,7,8,9))  ORDER BY v.weight = C:\utils\eclipseworkspace\civicrm-3.4\packages\DB\DataObject.php, backtrace, 2358
C:\utils\eclipseworkspace\civicrm-3.4\packages\DB\DataObject.php, _query, 1610
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\DAO.php, query, 147
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\DAO.php, query, 874
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\OptionGroup.php, executeQuery, 109
C:\utils\eclipseworkspace\civicrm-3.4\CRM\Core\PseudoConstant.php, values, 503
C:\utils\eclipseworkspace\civicrm-3.4\api\v3\Activity.php, activityType, 296
C:\utils\eclipseworkspace\civicrm-3.4\api\v3\Activity.php, _civicrm_api3_activity_check_params, 82
C:\utils\eclipseworkspace\civicrm-3.4\api\api.php, civicrm_api3_activity_create, 134
C:\utils\eclipseworkspace\wcan\drupal\modules\surveyactions\surveyactions.drush.inc, civicrm_api, 244
C:\utils\eclipseworkspace\wcan\drupal\modules\surveyactions\surveyactions.drush.inc, createActivitiesForGroup, 154
, surveyactions_fill_groups,
C:\utils\drush\includes\drush.inc, call_user_func_array, 51
C:\utils\drush\drush.php, drush_dispatch, 90
C:\utils\drush\drush.php, drush_main, 40
« Last Edit: July 22, 2011, 01:31:08 am by Eileen »
Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Re: Performance & the API (& caching that isn't)
July 22, 2011, 09:20:20 pm


Index: Activity.php
===================================================================
--- Activity.php   (revision 35439)
+++ Activity.php   (working copy)
@@ -293,7 +293,7 @@
 Unbelievable - this is why the caching wasn't working
 
 
Code: [Select]
   require_once 'CRM/Core/PseudoConstant.php';
-    $activityTypes = CRM_Core_PseudoConstant::activityType( true, true, true, 'name', true );
+    $activityTypes = CRM_Core_PseudoConstant::activityType( true, true, false, 'name', true );
 
     $activityName   = CRM_Utils_Array::value( 'activity_name', $params );
     $activityTypeId = CRM_Utils_Array::value( 'activity_type_id', $params );
Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute

Erik Hommel

  • Forum Godess / God
  • I live on this forum
  • *****
  • Posts: 1773
  • Karma: 59
    • EE-atWork
  • CiviCRM version: all sorts
  • CMS version: Drupal
  • MySQL version: Ubuntu's latest LTS version
  • PHP version: Ubuntu's latest LTS version
Re: Performance & the API (& caching that isn't)
July 25, 2011, 12:14:14 am
Nice catch!
Consultant/project manager at EEatWork and CiviCooP (http://www.civicoop.org/)

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Re: Performance & the API (& caching that isn't)
July 25, 2011, 01:33:05 am
Yep, not sure if because of this or not but I notice API tests have gotten worse recently.

I haven't been able to run them on my PC lately as they silently exit during the contribution test (I put in an actual test on this since it kept happening & logged it ).

Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute
Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • APIs and Hooks (Moderator: Donald Lobo) »
  • Performance & the API (& caching that isn't)

This forum was archived on 2017-11-26.