CiviCRM Community Forums (archive)

Hello there,

I had been experiencing slowness on a few different sites recently.  On two seemingly very similar sites on the same host, I was seeing one site out perform the other by a significant amount, and this was especially noticable / critical when it came to Ajax calls.

After looking at a simple javascript call with xdebug at the Albany code sprint, we worked out that loading the localisation files was putting a big strain on the server.

On a simple ajax call (which actually was returning an error message) FileReader->read had an 'invocation count' of 16813 and a 'total self cost' of 35%.

A proposed solution was to look into apc and making sure that the translation file (e.g. civicrm/l10n/en_GB/civicrm.mo is 901K) got cached with apc.

But this doesn't seem to work.  Seems like my apc is (probably reasonably) not caching .mo files.

I am wondering how I can get this file cached?

Or maybe there is a different approach / something else I am missing

I did see this post about caching the translation object in wordpress land and wondered if a similar approach could be taken with civicrm.  http://wordpress.org/extend/plugins/cache-translation-object/

My short term solution was to turn off translation.  That is an option for me because en_GB is not too different from en_US, but that is obviously not normally an option for other languages.

Thoughts very much appreciated.

Michael

might want to ping bgm on IRC since he seems to know a fair amount about localization and APC

lobo

Hey Xavier,

The particular problem for me was in Ajax autocomplete since this could often be called 15 to 20 times in a few seconds by a single operator (try and find 'McAndrew, Michael' or 'Third Sector Design'   Multiply that by a few people in an office trying to look for someone and my server began to not like it.

It seems like most autocompletes and a lot of ajax, if they are just returning DB data with name values wouldn't need any localisation.

I'll ping bgm and we can see what he says...

Michael

Hi,

I know that hartmut did investigate and found some tricks for big improvements. Can you ask him on IRC next time you see him?

Quote

Do we need localisation on the API calls?

The major pain is when the request gets generatered ~30 times in the space of a few seconds.  I guess you can't tell how someone is using the API and it would be good to turn it off anywhere when it is not necessary.  In other instances, it isn't an unbearable overhead, I guess.

Unfortunately I don't have much to add except that with the PHP "APC"  cache I do not have this performance issue (most of my sites are in French).

However, in Albany I checked on Michael's dev environment, and for some reason, APC was not helping at all. But in theory, and in my dev/prod environments, I measured with xdebug a clear improvement when APC is active. ACP keeps the ~ 1 mb file in memory and avoids loading it every time.

You also need to make sure your operating system has enough RAM to be able to cache the files. For Apache/MySQL, I never work with less than 1 GB of ram.

Xavier was asking if it's necessary to load the translation in API calls: the way the code works, it loads the translation file when ts() is first called. But it might be worth tracing why ts() was called for an auto-complete on a contact name.

In terms of bigger changes, I often wonder if a DB-based solution such as Drupal's is better (for example, translating CiviCRM extentions is not easy right now). Drupal loads only the necessary translations on each request, but it does a "select" query for each string. On the other hand, most of those queries get cached in MySQL. I rarely have seen people complain about Drupal's i18n performance because of those queries.

Hitting the ajax contact_get url causes crm_translate to be called 398 times.

Most of them seem to be as part of getfields which in turn calls $BAO::fields which turns out to be a fairly slow function as it calls 'ts' many times

It's not about language - the function still hits performance when your language is set to US English.

I haven't done much more than a quick look this time but I remember last time I tried changing to a custom translate function (replace crm_translate with something that did nothing) & got a boost on performance.