Author Topic: fetch multiple contacts by known ids (Read 4281 times)

pike67 · September 01, 2012, 01:27:50 pm

Hi

If I have an array of contact_ids, can i get all the contacts with those ids in one fetch ?

Some background info .. I need to list all organisations in a group *and* their representatives. 'is_representative' is a custom relation. ideally i would fetch everything in one go, but even if its possible i wonder if it would be efficient, under the hood. i can fairly efficiently get all the 'is_representative' relations with their source and destination ids, so I'm hoping with 3 fetches I can tie the lot together with PHP.

curious,
*-pike

pike67 · September 02, 2012, 05:47:30 am

... trial and lots of errors later ..

I tried filtering as suggested here
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/INRelationshipType.php

Code: [Select]

$params = array( 
	'id'	=> array( 
		'IN' => array('0' => 767,'1' => 2478)
	)
);
$results = civicrm_api( 'contact','get',$params );

didnt work for contact.id

I tried filtering as suggested here
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/filterIsCurrent.php

Code: [Select]

$params = array( 
	'filters' => array(
		'id'	=> array('IN'=>array('0'=>'2478','1'=>'2480'))
	)
);
$results = civicrm_api( 'contact','get',$params );

didnt work. also tried contact_id and contact_a_id and a few other variations, but i think the idfield is hijacked in utils.php::_civicrm_api3_dao_set_filter, and the 'IN' filter wont work on it. not sure. Actually not sure if it ever gets there, because I couldnt use the filter on any other field either.

I tried using a 'where' param too, but it didnt get picked up afaics.

Then digging through the source I came to this weird solution

Code: [Select]

$params = array( 
	CRM_Core_Form::CB_PREFIX.implode(',',array(2478,2480))	=> 'dummy'
);
$results = civicrm_api( 'contact','get',$params );

that works

It abuses a feature in CRM_Contact_BAO_Query::includeContactIDs that is probably ment for something completely different, but it works, so unless someone knows a better solution, I'll stick with it.

But .. if I start writing api calls like that, i would probably be better off writing SQL right away

Could I ? How ?

*-pike

Erik Hommel · September 03, 2012, 12:51:50 am

Ofcourse you could, but I would recommend using the API and fixing the issue. I think that you should be able to do an API call like you tried:

Code: [Select]

$params = array(
   'id'   => array(
      'IN' => array('0' => 767,'1' => 2478)
   )
);
$results = civicrm_api( 'contact','get',$params );

As you have been digging in the code it would be great if you could fix the API code?

And if you want to go directly to SQL, use

Code: [Select]

$result = CRM_Core_DAO::executeQuery(your query statement);
while ( $result->fetch() ) {
etc..

xavier · September 03, 2012, 01:26:24 am

Hi,

I'd suggest not to go the sql road, unless you are dead sure it's for a one shot and that you don't mind if it works or not after the next civicrm upgrade.

As for the api.contact.get, indeed that's missing and that has been requested and needed several time.

I would suggest keeping it simple (for the user) and have the id being either an integer (as now) or that can be an array

Code: [Select]

$results = civicrm_api( 'contact','get', array ('version'=>3, id=> array (767,2478));

pike, could you modify the function in api/v3/Contact.php ?

I think it will be needed as well (for smarty and ajax) to have something like id="767,2478" too (like we did for return), but that's syntax sugarcoating in api/api.php or utils probably

pike67 · September 03, 2012, 01:30:15 am

Hi Erik

thanks for your reply

Quote from: Erik Hommel on September 03, 2012, 12:51:50 am

As you have been digging in the code it would be great if you could fix the API code?

I would if I could. But I dont even grab the concept yet. I think the most
important fix at this point is the documentation.

Are you saying that this notation
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/INRelationshipType.php
and other operators ('=', '<=', '>=', '>', '<', 'LIKE',"<>", "!=", "NOT LIKE" , 'IN', 'NOT IN')
should theoretically work on all entities, all fields ?

This code is implemented in _civicrm_api3_dao_set_filter.
That is called by _civicrm_api3_basic_get, which is used by civicrm_api3_relationship_get.

But civicrm_api3_contact_get uses CRM_Contact_BAO_Query::apiQuery instead,
which doesnt call _civicrm_api3_dao_set_filter. Shouldnt it be there somewhere ?

$2c,
*-pike

Erik Hommel · September 03, 2012, 01:49:52 am

Yes, all API's should (as in this is the law

) work with the documented operators. Having said so, API maintenance is community work and sometimes some API's that are not used often might not have been updated. And there is always the possibility as a bug. The goal is still and will remain that all API's work in the same manner and the same rules apply. In the coming sprints some participants (me, Xavier, Eileen, Coleman and perhaps others) will give some tender loving care to API's.

I have not dug deeply into the Relationship API recently, but the last time I did there were also some inconsistencies in the BAO that forced some API behaviour.

pike67 · September 03, 2012, 02:30:33 am

Quote from: Erik Hommel on September 03, 2012, 01:49:52 am

Yes, all API's should (as in this is the law ) work with the documented operators.

OK. While we're at it, what is the difference with the 'filter' notation then ?

Quote

I have not dug deeply into the Relationship API recently, but the last time I did there were also some inconsistencies in the BAO that forced some API behaviour.

It seems RelationShip works just fine, its the Contact that is .. different.

*-pike

pike67 · September 04, 2012, 07:18:13 am

Quote from: Erik Hommel on September 03, 2012, 01:49:52 am

Yes, all API's should (as in this is the law ) work with the documented operators.

Yes, but several notations have been 'documented'. Actually, I think the filters notation is the more generic one. Why did you choose for the non-filter notation ?

It would be easy to implement the filters to contact by adding a call from CRM_Contact_BAO_Query::apiQuery to to /civicrm/api/v3/utils.php::_civicrm_api3_dao_set_filter.

However, that ties the non-api functionality into the api functionality. I think that is the wrong way around. It would be better to move /civicrm/CRM/Contact/BAO/Query.php::apiQuery into /civicrm/api/v3/Contact.php - it seems to deal with the api, so I dont think it belongs in the CRM part of the code ?

Since I have a working workaround, I'm not going to touch it, for now ...

$2c,
*-pike

Erik Hommel · September 04, 2012, 07:20:31 am

I will not go into all the details from your comments, great you are investigating in this depth! In short the answer is: we all do community work as much as we can........

xavier · September 04, 2012, 08:11:15 am

Quote

Yes, but several notations have been 'documented'. Actually, I think the filters notation is the more generic one. Why did you choose for the non-filter notation ?

Just to be sure: what are the others you found?
I'm afraid the best explanation is as often legacy. It started with a simple: you can add any of the fields as a filter, so get only the contacts that are individual with contact_type=Individual

Then Some "not exactly fields but close enough" came in, eg group=42 to get the contacts in that group
Then some "less and less fields", eg rowCount and sort

Then we decided that that was polluting the namespace and put that into option and filter

And then we don't want to break compatibility, so both are still valid and because our wiki versionning made the older versions better documented, well, old habits die hard

But yes, filter and options and the new black, do promote, they will need it

Eileen Erik and I are going to clarify the doc in the sprint after civicon. Feel free to join us (online is fine too)

Quote from: pike67 on September 04, 2012, 07:18:13 am

However, that ties the non-api functionality into the api functionality. I think that is the wrong way around. It would be better to move /civicrm/CRM/Contact/BAO/Query.php::apiQuery into /civicrm/api/v3/Contact.php - it seems to deal with the api, so I dont think it belongs in the CRM part of the code ?

I am not sure why it would be different for the apiQuery than the general query. Conceptually, shouldn't matter who is doing the query and could be the same function (in the BAO), and the query code belong to the BAO level IMO.

X+

pike67 · September 04, 2012, 08:32:58 am

Quote from: xavier on September 04, 2012, 08:11:15 am

Quote
Yes, but several notations have been 'documented'.

Just to be sure: what are the others you found?

These two, neatly in the same folder ;-)
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/INRelationshipType.php
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/filterIsCurrent.php

I'll assume filters is the way to go. I'm curious about 'options' - havent seen those yet.

Quote

Quote from: pike67 on September 04, 2012, 07:18:13 am
However, that ties the non-api functionality into the api functionality. I think that is the wrong way around. It would be better to move /civicrm/CRM/Contact/BAO/Query.php::apiQuery into /civicrm/api/v3/Contact.php - it seems to deal with the api, so I dont think it belongs in the CRM part of the code ?

I am not sure why it would be different for the apiQuery than the general query. Conceptually, shouldn't matter who is doing the query and could be the same function (in the BAO), and the query code belong to the BAO level IMO.

I agree. But /civicrm/CRM/Contact/BAO/Query.php::apiQuery builds on top of the BAO level, so it could still move to the api level.

My problem is, I think the api should sit on top of the core, and not be intertwined with it. The core should work without the api. if I call _civicrm_api3_dao_set_filter from /civicrm/CRM/Contact/BAO/Query.php, the two worlds would get mixed up. Am I making sense ? :-)

$2c,
*-pike

xavier · September 04, 2012, 08:52:21 am

Quote from: pike67 on September 04, 2012, 08:32:58 am

These two, neatly in the same folder ;-)
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/INRelationshipType.php
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/filterIsCurrent.php

I'll assume filters is the way to go. I'm curious about 'options' - havent seen those yet.

well, options are for stuff like limit and offset and sort.

When we went for filters we discussed using method overloading instead (so you can have date =xxx or date = array ('>=' ...

Don't recall what was the reason we went the filter way.

Quote

My problem is, I think the api should sit on top of the core, and not be intertwined with it. The core should work without the api. if I call _civicrm_api3_dao_set_filter from /civicrm/CRM/Contact/BAO/Query.php, the two worlds would get mixed up. Am I making sense ? :-)

The last one that suggested to separate the api from the core ended up nailed to the door, as an example. Are you really sure you want to pursue that quest?

Seriously, agree with you and it shouldn't have api stuff in the BAO. but
1) a method that starts with _ and considered private, you shouldn't call it anyway
2) unless you are developing for the core or the api, you should rather call the api instead of going to the BAO or sql level
3) some core is using the api "the proper way" eg the jobs are calling the api, or some UI is calling the API ajax, so in general, BAO shouldnt' call the api but that's normal that some of it uses the api (eg UI or batch job)

X+

pike67 · September 04, 2012, 09:00:00 am

Quote from: xavier on September 04, 2012, 08:52:21 am

The last one that suggested to separate the api from the core ended up nailed to the door, as an example. Are you really sure you want to pursue that quest?

.........ehm.............yes.

Quote

1) a method that starts with _ and considered private, you shouldn't call it anyway

Oh boy. Very true. That filter stuff should move up instead. Or as well. Or be duplicated. Hm.

$2c,
*-pike

Jeremy Proffitt · March 07, 2013, 12:33:50 pm

I submitted a patch (CRM-12051) that handles this issue in the BAO, checking if the appropriate value of the _params array in whereClause() is an array, and modifying the clause accordingly.

Eileen · March 07, 2013, 01:12:56 pm

I agree that patching in the BAO is the right place. At the api level the array + operator type approach is the more recent as it is more generic.

Code: [Select]

These two, neatly in the same folder ;-)
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/INRelationshipType.php
http://svn.civicrm.org/civicrm/branches/v4.2/api/v3/examples/Relationship/filterIsCurrent.php

I'll assume filters is the way to go. I'm curious about 'options' - havent seen those yet.

I think you have just set the field to be optionally an array without the 'IN'?

CiviCRM Community Forums (archive)

News:

Author Topic: fetch multiple contacts by known ids (Read 4281 times)

pike67

fetch multiple contacts by known ids

pike67

Re: fetch multiple contacts by known ids

Erik Hommel

Re: fetch multiple contacts by known ids

xavier

Re: fetch multiple contacts by known ids

pike67

Re: fetch multiple contacts by known ids

Erik Hommel

Re: fetch multiple contacts by known ids

pike67

Re: fetch multiple contacts by known ids

pike67

Re: fetch multiple contacts by known ids

Erik Hommel

Re: fetch multiple contacts by known ids

xavier

Re: fetch multiple contacts by known ids

pike67

Re: fetch multiple contacts by known ids

xavier

Re: fetch multiple contacts by known ids

pike67

Re: fetch multiple contacts by known ids

Jeremy Proffitt

Re: fetch multiple contacts by known ids

Eileen

Re: fetch multiple contacts by known ids