Groups and multisite: creating a new group takes a long time

[ken]

I just created a new group and it took ages.

I have a feeling that since each group is now has the "Domain" group as it's parent, there's a lot of re-calculation going on. I suspect it's because the cache of contact-groups is being updated. My concern is what will happen as the numbers of groups and contacts expands: will timeouts start occurring?

Before I invest too much time investigating, can anyone confirm if this is how groups and caches work?

Thanks!

[ken]

Could this be related to CRM-9969 "Smart group cache-build fails for extremely large groups" ?

[Eileen]

Hi Ken,

This is a problem that we experience. My performance focus for multisites of late has been on CRM-10667 - which was causing smart groups to be unecessarily resolved via expensive queries. I'm not sure what the impact of this fix will be on the group save slowness - but I am running a backport of it on our 4.2 sites  (the backport has the same effect as the full patch but without the code tidy up)

Patch is currently https://github.com/fuzionnz/civicrm/commit/a66fc0e35d84151e5587f192d0dfa1d96aa27bbf - however, I rebase our patches when we upgrade to keep them clean so the link could change.

The other performance gripe relating to this is that the group_contact cache for the parent group is filled for all child groups & then filtered on the way out - causing huge group contact cache tables. A few threads discuss this incl
#1 on this post i
http://forum.civicrm.org/index.php/topic,25286.msg106758.html#msg106758
#2 on this one  (#1 on this is fixed by CRM-10667)
http://forum.civicrm.org/index.php/topic,21775.msg91193.html#msg91193

[ken]

Thanks, Eileen!

[Eileen]

I am looking at this again. The problem is that the L1 group has so many groups under it that it takes ages for the L1 group to build.

I am not sure it's actually necessary to start the hierarchy at the L1 group or even have multisite turned on at the top level since
1) turning the multisite setting on primarily causes users to be added to the L1 group - which is not necessarily desirable
2) turning the multisite module on implements ACL permissions  - not required @ the top level

However, this still leaves the lower level domains

1) with multisite turned on all contacts are added to the relevant L2 group - so membership of the L2 group via other groups is not particularly helpful except where the hierarchy is really being used & an L3 group is in place.

2)  Any groups created on an L2 domain are created as children of the L2 group (as above). The benefit of this is that the group thus created is visible within that L2 group. The downside is that as per above the group contact cache is excessively rebuilt when there are lots of groups.

I'm thinking that it might be better to use a mechanism other than hierarchy (group type) to indicate that the group should be available on a given domain (ie. in the groupACL and only use hierarchy where relevant (per #1)

NB - it's possible to have multiple domains without either multisite enabled of multisite permissioning

[Eileen]

Ken - can you please look @

http://forum.civicrm.org/index.php/topic,28173.0.html

I have been thinking my way through this whole problem