CiviCRM search applying actions to incorrect resultsets?

[Chris Burgess]

This follows on from an already identified issue where the use of multiple browser sessions (via tabs or windows in the same browser) may lead to the incorrect modification of data in CiviCRM.

I saw an instance yesterday where an apparently similar issue happened for a user who knows to avoid working in multiple tabs or multiple browser windows. I should stress that despite throwing an hour and a half at trying to reproduce the bug on a test setup, I wasn't able to do so. However, the symptoms seem to match the same issue  pretty closely.

Here's what I know:

• A staff user created a group of three people (resultset A), then used CiviCRM's "Send email to contacts" feature to send them an email.
• They then typed up the body of the email and hit send.
• The email was sent to a number of users who were not in the initial group. The "Send email" activity was logged against some hundreds of users (resultset B).

In analysing the Apache logfiles from the user's session with CiviCRM, I see entries which indicate that the user had engaged some complex searches fifteen minutes before sending the mail, but that those results had not been returned. (Apache logs show requests with the timestamp the connection was initiated, and appearing in the order they are completed, so seeing requests in the order 13:44, 13:29, 13:27 indicates that the first two were slow to respond.)

Having seen similar behaviour when multiple or browser windows were open, I suspect that the same issue might have affected this user, but without even using multiple tabs. The user made a search (resultset B), then gave up on the results after a few minutes, and performed a different search (resultset A) to send the email. While the email was being typed up, the first search result (B) was returned. This search result was attached to the qfKey and then the email action was tied to the incorrect search result.

I want to reiterate that I haven't been able to duplicate this bug, but I'd like to hear from anyone who's seen similar things happen. I don't have the exact sequence of events to duplicate it ... it may be that B was returned before the email was composed, while it was being composed, or even after the email was composed (if the action is stored in the session along with the resultset?).

Any ideas?

[petednz]

Just flagging that unpredictable outcomes have also been reported when eg clicking on the A-Z options above a search result - have no idea if the trigger is the same - but the outcome is - with the potential disasterous consequence of someone aiming to 'delete' 5 records and wiping their entire contact list. http://forum.civicrm.org/index.php/topic,8536.0.html
so flagging it in case there are any common element to these - though of course the 'overlapping search' described by chris above could have been the culprit with these too.

[Donald Lobo]

argh

your theories seem to match with what happened. In most cases when u move to another page, the browser should drop the connection to the server and the mysql process should also terminate. I've seen the latter not happen a few times, but have assumed that the browser did drop the connection (but might have been a bad assumption)

lobo

[Chris Burgess]

Happy to provide access to a copy of the DB and logfiles for investigation if you want to try your hand at reproducing it. I'd love to be able to duplicate this. Does the browser really need to keep the connection open? I hadn't considered that - figured that CiviCRM was attaching the resultset to the session serverside, even after the browser was off doing other things.

Am I correct in thinking that this is basically the same issue as CRM-2876? If so, would assigning a unique qfKey to each generated form prevent this happening, or is it more complex than that?

Donald, you suggested that CRM-2876 might not get looked at until there was a switch in the underlying framework, but that isn't on the roadmap (only mention I can find is this). Do you think that's still the case?

[Donald Lobo]

seems like there was a zombie apache process which kept running even after the browser terminated the connection. this process probably updated session state when it was done, hence that behavior

if we generate a unique qfKey to each form session (i.e. CRM-2876) that will solve the problem. the main issue currently is this will result in a significantly increased session size, and the sessions is already quite bloated (IMO). Ideally we need to start moving form/page state away from the session into its own table(s)

lobo

[xavier]

Hi,

Yes, that's something I've experienced as well.

Why is this in the session anyway ? Shouldn't the search key (qfKey), or the search criteria, or the list of ids (for when you select a few contacts) be a parameter of the action form ?

Am I missing something ? qfKey is not the most documented part of quickform, not sure what it really does.

X+

[rimu]

Ideally we need to start moving form/page state away from the session into its own table(s)

Absolutely.

Session is very convenient, but it's dangerous to get seduced by that, because it leads to all kinds of bleed-through problems

[petednz]

http://issues.civicrm.org/jira/browse/CRM-5274 - and noting that on IRC lobo says he has already developed a configurable check to set maximum 'send an email' to 50 - will be available shortly

[AndrewM]

An issue analogous to the one discussed can be reproduced very simply:
choose 'Advanced Search', search for something, e.g. name is 'Miller,
%'. Then choose 'All contacts' on the results page, select 'Send Email
to Contacts'. When this page comes up, open a new tab, and do another
advanced search, for example, for 'Smith, %'. This is a fast-running
query, but it simulates a long running query having just finished.

Now go back to the first tab, and hit reload. You will see that you are
now about to send an e-mail to people with 'Smith' as their surname,
instead of Miller.

I have constructed the simplest possible patch which resolves this issue
on a test instance of CiviCRM I set up - when I refresh, the second
advanced search does not affect the results of the first search.

Patch:
diff -rbud /usr/src/civicrm/CRM/Contact/Controller/Search.php
/var/www/civicrm/CRM/Contact/Controller/Search.php
--- /usr/src/civicrm/CRM/Contact/Controller/Search.php 2008-12-12
09:00:35.000000000 +1300
+++ /var/www/civicrm/CRM/Contact/Controller/Search.php 2009-10-23
14:09:49.584374179 +1300
@@ -56,7 +56,7 @@
     function __construct( $title = null, $modal = true, $action =
CRM_Core_Action::NONE ) {
         require_once 'CRM/Contact/StateMachine/Search.php';

-        parent::__construct( $title, $modal );
+        parent::__construct( $title, $modal, null, null, true);

         $this->_stateMachine =& new CRM_Contact_StateMachine_Search(
$this, $action );

[Donald Lobo]

I've been thinking a wee bit about this, and i think in 3.2 we'll do the following:

1. move all civicrm form / page session storage to the civicrm_cache table

2. each page/form is independent and has its own entry in the cache table

3. the cache already has a created / expired date entry. So we can set them to autoexpire in 24 hours (config variable). Also a cron job can go in an clean all expired entries

4. modify quickform to use the above storage rather than the $_SESSION

I think this will have quite a few benefits and reduce session size

lobo