CiviCRM Community Forums (archive)

I am not sure if this is the appropriate forum as I did not see a performance related thread.

I am curious if Civicrm has the ability to set a throttle on how many transactions it can perform in any given second, or at least possibly put a cap to limit system and resource usage?

I have run into several instances where some large transaction such as removing 10k member accounts or similar has caused Civicrm (through Apache and Mysql) to consume pretty much all available server memory and spike the CPU upwards of 80% for extended periods of time.

I find that large cron jobs could some a similar impact on a smaller scale.

Is there a way to throttle Civi down, or lessen its impact on a server overall for large requests?

Thanks

I agree with Hershel -- there isn't a whole lot that Civi (internally) can do for resource limits, but you will find more throttling abilities in the web server, database server, and network layer. These are general performance management practices that apply to any PHP+MySQL web app. Things like:

 * Configure the Apache (and/or FCGI and/or nginx) thread pool -- things like max #connections, max connection time, max thread lifetime, etc
 * Configure the MySQL connection limits, cache sizes, etc
 * Set the PHP memory limit and PHP request time-out

Crudely speaking, this technique (setting caps or limits) can help you to *manage* performance better by trading one performance issue for another (hopefully less noticeable) issue -- for example, if you limit a transaction rate, then some transactions will be guaranteed CPU time, but other transactions won't be active at all -- they'll be canceled or queued. (Another example: you mention a "large transaction such as removing 10k member accounts" which causes CPU to spike. Limiting the #transactions won't affect the size of that-- it's still a big transaction that requires a lot of CPU.)

To actually improve performance, some other angles to investigate are:

 * Improve the hardware (more CPU cores, more iops, more RAM, etc)
 * Improve the software -- identify specific use-cases/page-requests/scripts/situations that cause high load, examine why those use-cases create high load, and implement faster algorithms