Smart Groups big bug???

[madronemedia]

http://vimeo.com/4405539

I could really use some feedback on this. I was working on a totally different project when I realized that something was wrong with the smart group I created. (Or, maybe my idea of what smart groups are supposed to do is wrong). Either way, any help would be great.

While creating a training video, and finding that the advanced searches are OR statements when checking more than one group, I read the documentation on search builder. I wanted to find all contacts that are members of Group 1 AND Group 2. I set it up, ran it, saved as smart group. Looks great. Just for fun I decided to open one of the matching contacts and remove them from one of the groups.

Uh oh.

Summary

If you change a contact's record so that they no longer would belong to a configured smart group, they are NOT removed from the smart group. If you search for members of the smart group they are still listed. Not good. I've tested this in the drupal online demo site and the rasa standalone demo and got the same results.

BUT, I found, by trial and error, if I click edit the search criteria after pulling up the smart group members, it will then correctly display the matching contacts. However, this doesn't stick.

Since we have vimeo and this would require me typing wayyyy tooo much. I documented the bug in a video.

http://vimeo.com/4405539

[madronemedia]

  An Update. Okay, I've read the smart groups documentation. Are smart groups a static or dynamic query? I assume dynamic because if I update a contacts record to fit the smart group criteria, they are added to the smart group. However, I'm still finding if I remove the criteria from a record they stay in the smart group.

I just found the "update smart group option". I tried this, was then asked to re-save the smart group, and low and behold, it is now correctly identifying the correct contacts.

So, why aren't smart groups smart? Someone who's used them more, please help me understand why we would need to even chose "update a smart group" to get the correct members to show.?

[petednz]

Hi Mark - nice use of vimeo

without stepping through the whole process my question is when you went back to members of the Smart Group and saw 3 - was that showing that there were 2 who were 'members' and 1 who was a 'removed member' - since the removal of someone from a group gets recorded as a new 'status' ie removed.

The crux is if within Group Members you searched for 'removed' would that 3rd person be listed - and if you actually use the group in CiviMail does it list that the mail is going to 2 or to 3 people?

[Donald Lobo]

I'm pretty sure this is a bug and its because we dont flush the smart group cache when the criteria is changed. Can you please file an issue, i'll fix it later today

for performance reasons, we cache the smart group results in a cache table. hence the smart groups dont appear smart when we hit a bug

lobo

[madronemedia]

Thanks for your feedback. I have submitted the bug report.

Mark

[fen]

Is there a way to manually flush the smart groups cache?  I'm seeing some very odd smart groups issues on sites upgraded from Drupal 5.10/CiviCRM v1.7 to Drupal 6.12/CiviCRM v2.2.6.

 1. From Manage Groups /civicrm/group and click on Dec 08 Mailing (Smart Group) >> Members link, I see 951 records

 2. From there, if I choose Edit Smart Group Search Criteria for Dec 08 Mailing I get 11 Group Members (the right number)

 3. I also get 951 records if I go to Advanced Search and choose Dec 08 Mailing

 4. If I recreate the search criteria (using the criteria displayed in step 2 above) and create a new group Dec 08 Mailing - QA1 everything works properly and shows 11 members.

There are too many groups to recreate them all by hand (hundreds spread over 15 sites).

Thanks!
=Fen

[Dave Greenberg]

Smart group membership is cached in this table: civicrm_group_contact_cache

Truncating the table flushes the cache - and I "think" any contact edit or insert also flushes and rebuilds it.

Check these links for related details:
http://forum.civicrm.org/index.php/topic,6375.0.html

http://issues.civicrm.org/jira/browse/CRM-3729

[fen]

Hrm.  I truncated civicrm_group_contact_cache and now:

 1. From Manage Groups /civicrm/group and click on Dec 08 Mailing (Smart Group) >> Members link, I now see 0 (zero) records

 2. From there, if I choose Edit Smart Group Search Criteria for Dec 08 Mailing I get 11 Group Members (the right number)

 3. I also get 0 records if I go to Advanced Search and choose Dec 08 Mailing

Do I need to do something to re-create the cache?  Is there some automated way to do this (perhaps a sql query) as I want to do this for *all* smart groups (assuming it works).

Thanks again!
=Fen

[Dave Greenberg]

Fen - I'm not too familiar with the cache functionality - but I did some greps on the codebase and learned the following:

* CRM/Contact/BAO/GroupContactCache.php contains the methods which flush and rebuild the cache

* The load method is invoked in CRM/Contact/BAO/Query.php - and the comments seem to indicate that searches trigger the rebuilding of the cache.

If you can catch lobo on IRC he can probably clarify - but he's in transit back from Europe so you might want to dig around in this part of the code.