CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • APIs and Hooks (Moderator: Donald Lobo) »
  • contact_create() API costly?
Pages: [1]

Author Topic: contact_create() API costly?  (Read 1009 times)

xcf33

  • I post frequently
  • ***
  • Posts: 181
  • Karma: 7
  • CiviCRM version: 3.3.2
  • CMS version: Drupal 6.19/6.20
  • MySQL version: 5.x
  • PHP version: 5.2.6
contact_create() API costly?
June 08, 2010, 09:21:01 am
I been running into PHP allowed memory exhausted fatal error lately running contact_create() call inside a for loop.


I been monitoring the memory usage of my script memory_get_usage()

and see that the biggest memory usage jump is between before the
Code: [Select]
$contact =&contact_create($param)

and after the call.


As matter of fact, the memory usage keep going up after each call and it seems they are not freed therefore resulting in the memory exhaustion error.

I played around with turning dupe_check off in the $param of the API call and the result was

with 'dupe_check' => TRUE, each API call increased my memory usage by average .019 MB per call
with 'dupe_check' => FALSE, memory usage increases by .010 MB per call

But I still don't understand why the API call keeps allocating memory and not freeing them, since it only returns the contact_id of the created contacts.

I've tried to kill all my variables when they are out of scope, etc with not much success.



Any help would be appreciated!





« Last Edit: June 08, 2010, 09:23:19 am by xcf33 »

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: contact_create() API costly?
June 08, 2010, 10:24:31 am

hey chang:

the main memory leak is from our good friend, the DB_DataObject library. For some  reason they decided to store all the mysql references in a static array. So unless you explicity free each DAO when u r done with it, the code will leak memory :(

To simplify things, we've built a wrapper function: CRM_Core_DAO::freeResult( );

import calls that after processing every row, and hence does not leak memory :) This is a pretty heavy handed solution and is used with care (i.e. you cannot have a partial query running at this time). We do need to change a fair amount of code to free the DAO objects ($dao->free( ) ) when we are done using it. This however is probably a long arduous task

based on your test below, we know that dupe check leaks a bit, so going in and fixing that is probably not too hard

ping me on IRC and we can figure out a workaround


lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: contact_create() API costly?
June 17, 2010, 02:22:03 am
Can we call the free in the contact_create function (or in all the _create function really) or that's too risky and will get side effects ?

X+
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: contact_create() API costly?
June 17, 2010, 08:20:14 am

no thats a bit risky, since we dont know whats happening around that call

a better option would be to make sure that most functions free their DAO objects if not being used

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Re: contact_create() API costly?
July 12, 2010, 02:40:59 am
Reading this and your blog - does it follow that wherever a function like this appears:

Code: [Select]
        $dao =& CRM_Core_DAO::executeQuery( $query, $params );

        while ( $dao->fetch() ) {
            $viewNote[$dao->id] = $dao->note;
        }
        return $viewNote;

It should be replaced by



       
Code: [Select]
$dao =& CRM_Core_DAO::executeQuery( $query, $params );

        while ( $dao->fetch() ) {
            $viewNote[$dao->id] = $dao->note;
        }
        $dao->free();
        return $viewNote;

Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • APIs and Hooks (Moderator: Donald Lobo) »
  • contact_create() API costly?

This forum was archived on 2017-11-26.