Problems wit activities that involve a large number of contacts

[Fabian_SYSTOPIA]

Hello,

we are experiencing some difficulties when using activities that involve a large number of contacts. The reason may be that CiviCRM shows the names of all contacts when displaying or editing an activity.

This does not really make sense for more than let's say 50 contacts - if you have 20k involved contacts you will see about 50 pages of names when viewing the activity (or you will get an error).

It may also lead to problems when displaying activities in a contact's activity tab. If there are several activities that involve a larger number of other contacts (e.g. activities indicating regular postal mailings), it can easily take 30-60 Seconds only to load the tab. Probably CiviCRM loads all the contacts involved with all activities in teh background.

When you try to edit an activity with a large number of contacts, CiviCRM tries to load all the names of contacts involved in the "with contacts" field. If the number is too large, CiviCRM crashes and does not show any names in the "with contacts" field. If you edit the activity anyway, all formerly involved contacts will not be connected to the activity anymore.

My Questions:

• Does anybody have the same problem and any kind of approach or solution to tackle these problems?
• Is there any other way to use Activities connected to a large number of contacts?
• Should I post an issue on JIRA?

One approach could be to only show 50 contacts an add a link like "show all contacts" - however I have no idea how complicated that would be.

Looking forward to hear from you.

Best Regards
Fabian

[Dave Greenberg]

I think some organizations who have activities w/ large numbers of contacts use the new-ish option to "Create a separate activity for each contact" when creating the activity (this was requested / funded by the New York State Senate which has large datasets).

Have you considered using this feature for your use case(s)? It would definitely 'fix' the view/edit problems. Screenshot attached ...

If there's reasons why this is not a good approach for you, I think we could look at a core code change which would handle the 'With' (target) field as read-only if there are more than $N target contacts with a link to 'show all'. We'd need to think about whether it's necessary to modify the set of target contacts in this scenario (e.g. add or remove some) - which of course would add complexity.

[Fabian_SYSTOPIA]

Hi Dave,

thanks for the fast reply ^^

In our oppinion, both creating a separate activity for each contact and linking many contacts to one activity can make sense, depending on the activity:

• if you create an activity for a large postal mailing it makes sense to use one activity
• in case you want to call 50 donors it makes sense to use separate activities in order to be able to track the progress individually

Of course we could start to only use separate activities. This would imply to change old mass activities - not nice but probably possible. But I am not to sure if we would run into other problems in the long run:

Our customer does a monthly postal mailing to about 50k contacts. If we create separate activities we are talking about half a million activities per year only from those postal mailings.

Would you think that a "common" installation of CiviCRM is able to properly handle millions of activities? Do you know about details or experiences of anybody (e.g. the New York State Senate) that has used large numbers of separate activities?

If CiviCRM has no problems handling a large number of activities one could stop using mass-activities within CiviCRM alltogether. If both single and mass activities should be available in the future I think that it would be necessary to change the code in the way that you suggested.

Thanks again - greetings from Germany
Fabian

[lcdweb]

I think improving the UI when you have a single activity with multiple contacts is a worthwhile improvement -- regardless of whether you consider using the separate activities option.

We haven't really run into issues at the Senate, but that's likely just because activities are generally used with smaller contact sets. For the situation you are describing, where you are seeking to record a postal mailing, it does make more sense to handle that with a single activity record, and so a UI improvement like this would be very useful.

As for scalability -- i'm guessing your activity searches would get slower and more cumbersome if you are creating 50k activities at a time. But retrieving the records within a single contact record would be faster, as you don't have a large join with the activity_contact table. But that is just speculative theorizing. I think the question should be answered by thinking about how you want the data structured.

[Fabian_SYSTOPIA]

Thanks for the feedback. The more I think about it, the more I am convinced that improving the UI when you have a single activity with many contacts would make a lot of sense. As described above, both creating a separate activity for each contact and linking many contacts to one activity can make sense, depending on the task at hand.

In addition, when using separate activities for a large number of contacts, there is no good way of updating the activities. For example, if one schedules an actvity for 2000 contacts and want to set it to completed or cancelled two weeks later one would need to use batch update and can only update 100 activities at a time.

IMHO it would be very important, not to show more than 25 (50?) names when viewing or editing an activity (especially in the activities' tab of a contact). Adding or removing one or several contacts to an existing mass activity would of course be nice but not as urgent. However it would be important to make the 'With' (target) field read-only if there are more than $N target contacts (as described by Dave).

[Dave Greenberg]

Fabian - It seems like there's some decent consensus on improving the handling of the Target Contact field (read-only, limited list) for activities w/ more than N targets. I'll contact you directly about resources.

[artfulrobot.com]

This seems to be more of an issue in 4.5 with Select2 in use because it crashes the browser! I'm trying to record an activity against 5000 contacts but the form just crashes :-/

[Fabian_SYSTOPIA]

In case you have not found them already you may want to check out those posts:

http://forum.civicrm.org/index.php/topic,31488.msg139799.html#msg139799
http://forum.civicrm.org/index.php/topic,33425.msg141841.html#msg141841

Regards
Fabian

[artfulrobot.com]

To complete the thread. I reported this as a bug, and it got squished (thanks Coleman!).
https://issues.civicrm.org/jira/browse/CRM-15472