Author Topic: Stress testing (Read 3755 times)

JoeMurray · May 03, 2008, 01:25:36 pm

As a result of intermittent errors when running CiviCRM on a 256MB VPS which I suspect may have been due to php memory issues, I started wondering about the best way to do some stress testing of CiviCRM.

One very crude test that my support folks at RimuHosting (they're great, btw) did for me quickly on a site to be used for event registrations was to send multiple concurrent page requests and see what happened. Here are the results of searching for events, which involves multiple joins, versus searching for nodes in drupal. Note that the php error reporting may have been set at strict, which would have meant heavy logging on the civicrm page (that was happening on another vps at the same host with the default php.ini settings).

Quote

With stock Drupal 5.7 and CiviCRM 2.0.2, I managed to get my VPS to swap out in about 2 seconds using ab to benchmark civicrm from my home DSL connection, and note the number of failed requests:

Document Path: /drupal/civicrm/event
Document Length: 12044 bytes

Concurrency Level: 20
Time taken for tests: 70.411921 seconds
Complete requests: 50
Failed requests: 45
(Connect: 0, Length: 45, Exceptions: 0)
Write errors: 0
Non-2xx responses: 25
Total transferred: 302212 bytes
HTML transferred: 279108 bytes
Requests per second: 0.71 [#/sec] (mean)

Enabling APC stabilized things and the same request completed at about 5 requests per second. Not blazing fast, but the VPS still had half it's memory free versus being totally swapped out and killing processes.
Admittedly that concurrency level is a bit high, but I wanted to push things a little.

Comparing civicrm to a stock drupal 5.7 on the VPS showed a substantial difference. With 2000 test nodes, drupal completed the same test (concurrency of 20, 50 requests each) at a respectable 25 requests per
second:

Document Path: /drupal-5.7/node
Document Length: 14260 bytes

Concurrency Level: 20
Time taken for tests: 2.9899 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 742650 bytes
HTML transferred: 713000 bytes
Requests per second: 24.88 [#/sec] (mean)

A better stress test, as Lobo pointed out, would involve doing actual workflow expected on the site, like viewing events and registering for them. I started wondering what would be a good test harness for this.

Would Selenium be appropriate? Are there other better approaches?

Thoughts and suggestions would be welcome.

Donald Lobo · May 03, 2008, 01:45:46 pm

For any high volume installation, I would definitely recommend using memcached to cache some of the pseudo constant tables in CiviCRM (tables/values for country, state, gender, event type etc). CiviCRM is quite configurable, which also means we need to make a db call to retrieve all this information. this quickly adds up and results in probably more than 50% of the trips to the db to retrieve these "constant" values. also ensure that your mysql query caching is turned on.

We deployed a memcached version for the quest scholars project last year quite successfully. there is support in core to enable mysql, however we have not tested it since v1.8 (when quest was deployed). So you'll need to devote some developer resources to get this back on track.

lobo

JoeMurray · May 04, 2008, 03:15:29 pm

Very useful reply, Lobo - thanks.

xavier · May 05, 2008, 02:12:42 am

Hi,

Could you give some details about your memcached config ?

Otherwise, and forgive in advance stupid comments from someone that hasn't looked at that part of the code, some might already be implemented. What about caching at the php level these "constants" ?

eg. generating either a

cache/constants/country.php

Code: [Select]

<?php
$event_type[0] = 'Meeting';
...
$event_type[42] = 'Baby seal swimming session';
?>

or serialises it and store the serialised object (was faster when we tested, years ago...)

And obviously on the include constant, test if the cache exists, and generates from the db if it doesn't.

X+

JoeMurray · May 05, 2008, 05:05:05 am

Having now introduced myself to memcached, I see that it is not an independent technology underneath an app, but requires support in code. From your comment below, I'm guessing that what would be required is to check how the memcached code from 1.8 works, and what tables it is set up for, then track down changed references to those pseudo constant tables in 1.9-2.1 code. Probably also add support for any new pseudoconstant tables...

There are a few quest files in the tarball which always puzzled me...like quest.css. Does 'support in core' mean that most of the memcached code is in core or in quest files in the repo?

What are the downsides to enabling memcached be default? Are their signficant setup issues, or is just that no support had been developed for postgres?

Interested in the response to xavier, too.

Donald Lobo · May 06, 2008, 12:58:50 am

There are a couple of issues with the current memcached implementation

1. this was designed and built in 1.7. It has not been kept current (though i suspect this is not too much work). Most of the code is in CRM/Utils/Cache/Memcache.php. If enabled, all the pseudo-constant code (in CRM/Core/PseudoConstant.php) check the memcached cache before hitting the db

2. The code does not deal with or address cache invalidation. I would consider this an incomplete feature. The client we did this for did not modify the "pseudo constants" once we rolled it out. Hence we could live with this limitation

3. Generating a PHP constants file is probably a good idea. We should fix point 2 above, before we do this step

4. the memcached stuff is part of the core distribution

5. See point 2 above for downsides for memcached, i.e. its incomplete

6. most pseudo constants should support this (since most of them use the same helper fucntion)

7. not sure what the postgres connection is. Does civicrm work under postgres?

lobo

CiviCRM Community Forums (archive)

News:

Author Topic: Stress testing (Read 3755 times)

JoeMurray

Stress testing

Donald Lobo

Re: Stress testing

JoeMurray

Re: Stress testing

xavier

Re: Stress testing

JoeMurray

Re: Stress testing

Donald Lobo

Re: Stress testing