CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • Scalability (Moderator: Donald Lobo) »
  • Smarty caching efficiency
Pages: [1]

Author Topic: Smarty caching efficiency  (Read 3501 times)

ChrisChinchilla

  • I post occasionally
  • **
  • Posts: 104
  • Karma: 3
  • CiviCRM version: 4.1.2
  • CMS version: Drupal 6 & 7
Smarty caching efficiency
March 06, 2014, 03:30:56 pm
Hey everyone,

We keep hitting a major issue with smarty when it brings our servers to a grinding halt on cache writes (as it writes to the file system), we have quite a high traffic site, but I would think others have experienced this, over the next few days we are looking into some ideas to solve this, but i had a few questions:

  • Is this a problem with Smarty of CiviCRM's implementation of Smarty?
  • For our own usage, we're thinking of trying to redirect the file write to write to Redis instead, has anyone else tried something like this?
  • Any other thoughts or ideas?

Anything we come up with will be contributed back if it can be.

Thanks, Chris
Melbourne CiviCRM meetup group - http://www.meetup.com/MelbourneCiviCRM/

totten

  • Administrator
  • Ask me questions
  • *****
  • Posts: 695
  • Karma: 64
Re: Smarty caching efficiency
March 06, 2014, 04:08:21 pm
1. It's not supposed to write frequently. It's only supposed to write when you first use a (new/modified) *.tpl file. It sounds like maybe the system is generating false cache-misses on a systemic basis?

2. Storing in a redis is an interesting idea. I don't have any experience with redis+php, but my gut says you'd get the same benefit by just turning off Smarty caching entirely. The general goal of Smarty's caching system is to compile the *.tpl down to *.php and then allow the PHP runtime to apply furter optimizations (eg XCache, APC). If you have Smarty compile the *.tpl down to *.php code and then store the *.php code in Redis, then you would still need to have the PHP runtime evaluate the code. Short of writing a PECL extension or a core PHP patch, I think you'd wind up with something like this pseudocode:

Code: [Select]
$cachedTpl = $redis->get('the-compiled-template');
if (!$cachedTpl)
  $cachedTpl = $smarty->compile('the-raw-template');
  $redis->put('the-compiled-template', $cachedTpl);
eval("$cachedTpl");

Running through eval() is probably going to nullify much of the benefit of Smarty's caching architecture.

If the redis arrangement does help, it's probably because you're skipping the disk IO during false-cache-misses. But disabling caching entirely would produce the same benefit.

3. Can you determine if it's writing all templates -- or specific templates?

4. There are a few possibilities that come to mind:

4a. If there's some other process (eg a cron job or custom module) which flushes the cache frequently, then that would undermine the Smarty caching. You might investigate this adding some logging/monitoring to CRM_Core_Config::cleanup().

4b. A year or so back, I heard about an interesting case with CiviMail -- if you enabled Smarty support, it would compile a slightly different Smarty template for each outbound email. Micah was working on it at the time; I don't know if it was resolved.

4c. I think Smarty relies of file timestamps to assist in deciding whether to use a cached *.php or recompile the *.tpl to *.php. If there's something weird with the clock or filesystem, it could theoretically break the caching behavior.


ChrisChinchilla

  • I post occasionally
  • **
  • Posts: 104
  • Karma: 3
  • CiviCRM version: 4.1.2
  • CMS version: Drupal 6 & 7
Re: Smarty caching efficiency
March 06, 2014, 04:25:32 pm
I may have worded my original post slightly poorly, it doesn't write all the time / frequently, but when it does it hits server performance massively. So our main reasoning is stopping it writing cache to disk completely, like many other cache systems and have it write to memory instead.

This is sort of what you've touched upon in your very useful comments. i guess in summary our desire is to make the caching work better at the scale we're running things, maybe our initial thoughts are not the best way to achieve this, but I will investigate further and keep you posted.

Thanks, Chris
Melbourne CiviCRM meetup group - http://www.meetup.com/MelbourneCiviCRM/

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Smarty caching efficiency
March 06, 2014, 04:42:50 pm

i think micah and the eff guys put the smarty template cache directory in a memory file system.

no mods of code required and u get a fair speedup :)

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

ChrisChinchilla

  • I post occasionally
  • **
  • Posts: 104
  • Karma: 3
  • CiviCRM version: 4.1.2
  • CMS version: Drupal 6 & 7
Re: Smarty caching efficiency
March 06, 2014, 05:29:55 pm
Cool! You know if they documented it anywhere?
Melbourne CiviCRM meetup group - http://www.meetup.com/MelbourneCiviCRM/

herb

  • I’m new here
  • *
  • Posts: 27
  • Karma: 2
Re: Smarty caching efficiency
November 23, 2015, 02:02:19 pm
Totten, you mention ChrisChinchilla may be better off just turning of Smarty caching entirely. Just how would this be done? I'm trying to debug an issue and would like to see how it runs if the templates aren't compiled.

Thanks!
Web Developer
Freeform Solutions

totten

  • Administrator
  • Ask me questions
  • *****
  • Posts: 695
  • Karma: 64
Re: Smarty caching efficiency
November 23, 2015, 04:27:01 pm
If you're talking about my point #2... that was just a general idea. Never tried it and don't know steps. In skimming Smarty.class.php, the closest option I can see is "$force_compile" -- which doesn't sound quite like what I had in mind, but maybe it's what you had in mind. One would generally change/override options in Smarty by patching CRM/Core/Smarty.php or using hook_civicrm_config.

If you're talking about point #4b, IIRC CiviMail+Smarty is disabled by default. It's controlled by a define() in civicrm.settings.php.

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion »
  • Scalability (Moderator: Donald Lobo) »
  • Smarty caching efficiency

This forum was archived on 2017-11-26.