CiviCRM Community Forums (archive)

Hi,

We have been looking at moving to a format of holding 'special' params in one of 3 arrays

'options'
'filters'
'return'

Where options might contain

'limit'
'sort'
'offset'

Filters contain 'custom' filters - e.g. 'is_full' for events

Return should contain return what you want returned.

I need to check where we are with this as it won't work on all API - which one are you using? Here's an example

http://svn.civicrm.org/civicrm/branches/v3.4/api/v3/examples/OptionValue/SortOption.php

The alternate syntax might be $params['isCurrent'] but we decided we wanted to distinguish between 'real fields' & various special bits & pieces. There is a thread somewhere. Getting tests in place is the main bottleneck here.

Here is an example of the test for 'limit'

        /**
     *  Test limit param
     */
     function testGetOptionValueLimit()
     {
         $params = array( 'version'         =>  $this->_apiversion,);
         $result =& civicrm_api('option_value','getcount',$params);       
         $this->assertGreaterThan(1, $result, "Check more than one exists In line " . __LINE__ );
         $params['options']['limit'] = 1;
         $result =& civicrm_api('option_value','getcount',$params);   
         $this->assertEquals(1, $result, "Check only 1 retrieved " . __LINE__ );
 
     }

Ideally we would figure out how to make an automated test on this that checks all API. The complexity is the data (we'd need to be able to easily ensure each API had more than one entry)

Just to clarify - every API should be able to return more than 25 - unfortunately the syntax in inconsistent (and since we are really far into APIv3 I suspect we'll have to consolidate on supporting ALL of them).

Currently more than half the API support (all the ones that use basic_get or set_filter)

'options' => array('limit' => 200)

(the same ones also support option.limit => 200 from memory - there is a test / example somewhere)

There are a few that support 'rowCount' => 200 (contact_get, contribution_get, pledge_get and a couple of others)

From memory website_get Doesn't deal with limits of any type - I want to replace it with 'basic_get' but need a test first

Actually - it should be modified to use the same conversion function as pledge does & that should support both formats

http://issues.civicrm.org/jira/browse/CRM-9110

Yes, the difficulty of the different formats in different frameworks is why we got into multiple types. In fact I don't think supporting multiple types is the biggest difficulty - we need to funnel all use of these options through 1 or 2 functions so that whatever we do support is applied to all. Working through those changes (& ensuring tests are in place) is the biggest challenge here

What I do now if I need to process (say) 20% of 100K contacts, I perform a pure SQL query that returns the contact_ids of the group I want and then use the API to get each Contact separately.  This seems efficient enough to me, but my concern is that I try to minimize such pure sql queries and use the API whenever possible.  With offset and limit I could process contacts in batches of (say) 25 which is better than pulling the full contact records for all 20,000 in one call.  But if this is known to leak memory, then I'll stick with the pure SQL method and just be sure to document the dependencies.