Some custom groups not available in Views

[unclejustin]

Drupal 7.17
Views 3.5
CiviCRM 4.2.0

Trying to build a report using views. Went to add fields and could only see 2 out of 14 custom groups available. After some poling around I narrowed the issue down to this code beginning on line 415 in CRM\Core\BAO\CustomGroup.php in getTree() which is called from civicrm_views_custom_data_cache() in drupal\modules\views\civicrm.views.inc on line 307

Code: [Select]

$cache = CRM_Utils_Cache::singleton();
    $groupTree = $cache->get($cacheKey);
   
    if (empty($groupTree)) {

This is all called from a loop in drupal\modules\views\components\civicrm.core.inc on line 2281

Code: [Select]

while ($dao->fetch()) {
    // call getTree using $dao->id as groupID and $dao->extends as entityType

    $data = civicrm_views_custom_data_cache($data, $dao->extends, $dao->id, $dao->extends_entity_column_value, $dao->style);
  }

If I set $groupTree to NULL right before the if call (effectively getting rid of the cache check) then the problem goes away. I can now see and use fields from all 14 custom groups. Seems like the way that the views include file is calling this function in a loop won't work since the cache is set on the first iteration.

It's beyond my skills to try and really address the issue, I just stumbled on to the problem and found a hack way to work around it. Anyone else have a better idea what might be going on here, and how I could better fix it?

[petednz]

Yes I have been hitting this too but having just been using drupal 'clear caches' to get any new Custom Field Groups - a coded solution would be preferable. Let's see what others say to your hack

[Donald Lobo]

seems like we should just implement hook_civicrm_post on the custom_field and custom_group tables and invalidate the views cache, similar to:

civicrm_civicrm_postProcess in drupal/civicrm.module

lobo

[unclejustin]

I'll give those hooks a shot Lobo. Peteddnz, clearing the caches did not work for me, weird. Thanks for your ideas guys!

[unclejustin]

Tried invalidating views' cache, but that doesn't work. I didn't think it would since views' cache is not the problem. It has to do with how views integration is calling a function that is caching its results. getTree does a check to see if its data has been called before, thinks that the same request is being made, and returns the cached result. I don't think views integration can use getTree as it is currently written.

I'm able to make it work very easily, while retaining getTree's caching by adding a parameter to the function call: $ignoreCache = FALSE. Then checking for that flag to bypass the cache check if needed.

Code: [Select]

if (empty($groupTree) || $ignoreCache) {
This lets getTree be used by existing functions as it was, while allowing the rest of the code in civicrm.views.inc to remain unchanged.

[Chasen]

Hi guys,

I'm keen to implement this patch for my site as I'm experiencing a similar custom groups/fields issue (http://forum.civicrm.org/index.php/topic,26981.0.html)

problem is the site is using 4.1 civicrm rather than 4.2 and the information posted by unclejustin doesn't seem to apply 1:1 with the 4.1 files so would anyone be able to help a non-coder with a translation for 4.1 code? (note that upgrading to 4.2 isn't on the cards for the client as yet)

Thanks so much

[Swati Karande]

Hi,
I have attached a patch which is for clear a views cache for custom groups / fields.
I hope it will help.

[unclejustin]

Chasen try stepping through getTree in PATH_TO_DRUPAL/sites/all/modules/civicrm/CRM/Core/BAO/CustomGroup.php

I actually went in and started dumping variables to watchdog to see what was happening with my issue. That's how I found that the iteration was skipping over some of my custom groups. Get in there, start dumping and see what's happening. You may have a completely different problem than I was having.

[Donald Lobo]

swati:

i've applied the patch.

But can u go and see if u can reproduce this issue with the patch? A few folks have commented that clearing out the cache did not help

thanx

lobo

[Chasen]

FYI I've tried the patch suggested by unclejustin (who was kind enough to PM me the steps) but my custom data fields are still dropping from views.

Interestingly, before I upgraded to 4.2.6 (from 4.1) I could see the custom field group in the drop down list in views, even when the field wasn't showing up in the view result. Since upgrading I sometimes can, and sometimes can't.

It seems like views is letting me add the custom fields (based off cache?) which shows up results normally on pages where I'm using the fields, and then randomly the fields don't show up on my pages (doesn't line up with cron or flushing caches, i know that much) and when I check the view I get the broken relationship notice and when I click the add field dropdown to add the custom field back into the view I cannot see the custom group.

Furthermore, I have three custom groups and this is only happening to one of them. The only difference between them, that I can see, is the group that's being 'lost' is a group shared by two different user roles, where as the other two groups are only used by one role, respectively.

[Tiffani Chin]

Chasen -

Did you get this figured out?

Having the same problem -- I applied the batch suggested and the fields stuck around for several days and now they're gone again.

Would love to hear it if you have a solution!
-tiffani

[Chasen]

Funnily enough, I was just about to update this post.

For some reason, I lost a bunch of custom fields from my views and unrelated to that I re-enabled anonymous caching and for some reason the fields all came back!?


I have absolutely no idea why but when I enabled anonymous caching of pages the views show the custom fields fine, but when I disable Drupal caching they disappear?

Weird...

[Tiffani Chin]

That's interesting.

We don't have anonymous users on the site, so I don't cache for them. But I'm try to enable.

If it works, I'll take it.

Thanks for the update!
-tiffani

[petednz]

Yes I have been hitting this too but having just been using drupal 'clear caches' to get any new Custom Field Groups

Does this not work for you?

[Tiffani Chin]

No, clearing caches has not consistently worked for me.

However, I just upgraded the site to 4.2, I declined to save the changes that had dropped my custom fields from my views, and the custom fields were back.  And it seems like they are staying (we'll see how permanent it is). 

So, I have enabled caches for anonymous users and, with 4.2, seem to be keeping my fields. 

I'll post if I have any updates.

(Now, I just need to figure out why the custom fields are passing labels instead of values!)