CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Drupal Modules (Moderator: Donald Lobo) »
  • Custom data broken in Views
Pages: 1 [2] 3

Author Topic: Custom data broken in Views  (Read 5940 times)

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 12, 2010, 06:27:50 pm
Quote from: Eileen on April 12, 2010, 06:26:28 pm
Hi, I let Michael know you were looking for help on this so hopefully he'll be able to give some more direction as he knows this code quite well

cheers!

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 12, 2010, 06:31:13 pm
also, this file: civicrm_handler_field_custom.inc seems to be only about activities? Should it be renamed?

I'm going to leave it there for the moment - I'm just confusing myself and not really helping.

mdance

  • Guest
Re: Custom data broken in Views
April 13, 2010, 06:20:12 pm
Hey Datakid & Eileen,

It has been a while since I have dealt with views and civicrm but from what I recall my case was a bit different, the problem I was having was the data was not being formatted correctly so I had to override the views formatter so that I could reformat the output properly.

It sounds like your issue is that the civicrm custom data is not showing up in views in the first place.  I would check to make sure your settings.php aliases include all the civicrm custom tables, which it sounds like you've already done...

I would check the issue queue for the civicrm views integration to see if it has been resolved.  Unfortunately I don't have a good enough grasp of the civicrm custom data internals to fix the views integration.

I have attached a test module outlining how to override the views formatter, but I dont think this is what you are looking for.  You will have to rename the .txt to .zip

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Re: Custom data broken in Views
April 13, 2010, 06:22:21 pm
Thanks for replying Michael

I think the problem is likely to be that the handlers haven't been written for activity custom fields.

Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 15, 2010, 04:58:15 pm
Yes, thanks for the reply Michael. Unfortunately the test.zip.txt seems to be corrupt? I can't extract it (after removing the .txt on the end) with an error message that says same. In the meantime, I'll keep hunting

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 15, 2010, 06:32:05 pm
Something strange just happened:

decided to test across civi data and their types

created the following custom data sets:
NameUsed ForType
contact_testAll Contact TypesAny
activity_any_testActivitiesAny
activity_meeting_testActivitiesMeeting
event_any_testEventsAny
event_conference_testEventsConference

Without adding anything to settings.py settings.php, I went to views->list and chose to edit the view I'm struggling with.

Add a field, and "CiviCRM Custom: activity_any_test" appears. Despite not being added to settings.py settings.php

I am going to add all of the tables to settings now to see which others appear or don't appear.

=== Results ===

After adding the 5 Custom data sets listed above to settings.php, when creating views that apply to each data type, the results are:

NameUsed ForTypeDid it appear?
contact_testAll Contact TypesAnyYES
activity_any_testActivitiesAnyYES
activity_meeting_testActivitiesMeetingNO
event_any_testEventsAnyYES
event_conference_testEventsConferenceNO

I guess the thing I'm struggling with here is that there are "types" of Activities and Events (email, phone call, conference, meeting), but the Contact isn't typed. IE, I know there's a problem with the rendering of subtyped types, but I have nothing to measure the code...will keep searching.
« Last Edit: April 15, 2010, 07:22:04 pm by datakid »

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Custom data broken in Views
April 15, 2010, 09:36:26 pm

1. i suspect your civicrm and drupal db are the same. in which case you dont need to add anything in settings.php

2. as mentioned in IRC, your best bet is to add some debugging code to:

civicrm_views_custom_data_cache in drupal/modules/views/civicrm.views.inc

that will probably give you an idea as to why those tables are not exposed to views

3. if a contact is not typed, it applies to all contacts (note that there is just one civicrm_contact table), so basically that is exposed

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 15, 2010, 09:44:20 pm
Quote from: Donald Lobo on April 15, 2010, 09:36:26 pm

1. i suspect your civicrm and drupal db are the same. in which case you dont need to add anything in settings.php


No, they are different.

Quote from: Donald Lobo on April 15, 2010, 09:36:26 pm
2. as mentioned in IRC, your best bet is to add some debugging code to:

civicrm_views_custom_data_cache in drupal/modules/views/civicrm.views.inc

that will probably give you an idea as to why those tables are not exposed to views

Yep, looking into it now, thanks.

Quote from: Donald Lobo on April 15, 2010, 09:36:26 pm
3. if a contact is not typed, it applies to all contacts (note that there is just one civicrm_contact table), so basically that is exposed

lobo


thanks...

DanilaD

  • I post occasionally
  • **
  • Posts: 93
  • Karma: 11
Re: Custom data broken in Views
April 16, 2010, 10:35:25 pm
Datakid, looks like you are the first one to start heavily using custom fields with Activities and Views, as that line "return t('State/Province');" clearly should not be located in civicrm_handler_filter_activityType.

As a suggestion to further steps for you, I could recommend to check the database structure in http://wiki.civicrm.org/confluence/display/CRMDOC/CiviCRM+ERD+2.0, see how existing fields (like events) are handled in code and what it corresponds to on those maps, and make sure you have relevant bits of code for activities everywhere other data types get their bits of code. This approach helped me long time ago when working with relationships.

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Custom data broken in Views
April 17, 2010, 07:08:24 am
Quote from: datakid on April 12, 2010, 06:27:31 pm
ok, this file: civicrm_handler_filter_activityType.inc references State/Province:

39     function title( ) {
40         return t('State/Province');
41     }

which seems weird to me? I don't even know if this is where the problem is - I was just reading the source.

fixed for 3.2

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Custom data broken in Views
April 17, 2010, 07:10:13 am
Quote from: datakid on April 12, 2010, 06:31:13 pm
also, this file: civicrm_handler_field_custom.inc seems to be only about activities? Should it be renamed?

I'm going to leave it there for the moment - I'm just confusing myself and not really helping.

the comment was wrong here, fixed :)

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 18, 2010, 06:21:40 pm
Hoopla! We have a winner!

Ok, so it goes like this:

1. Complain a lot on forums and irc. Until someone pays attention, dlobo and eileen, cheers cheers to dlobo and eileen for answering my incessant questions.
2. As requested, debug via civicrm.views.inc, looking at the function: civicrm_views_custom_data_cache
3. Don't know where to start, try by outputting values to see if they match expectations. Add code at 4730-4732 as per:

Code: [Select]
4726     else {
4727         $tree = CRM_Core_BAO_CustomGroup::getTree( $entity_type, CRM_Core_DAO::$_nullObject, null, $groupID);
4728     }
4729
4730 /*   echo "entity type  = $entity_type ||\n";
4731  *   echo "subtype = $subType ||\n";
4732  *   echo "GroupID = $groupID ||";
4733  */
4734     switch ($entity_type) {

4. See output is as expected. Well, as much as I know what to expect. See first attachement "first_debug_attempt.png". In particular, we can see that my activity with subtypes (group id 6, cut off in pic) exists, and those subtypes are listed (23, 1, 24, 2, 25)

5. Jump down a  few lines to within the foreach loop and try again:

Code: [Select]

4765         $data[$currentgroup['table_name']]['table']['group']  = t('CiviCRM Custom: ') . $currentgroup['title'];
4766
4767         echo $currentgroup['title'];
4768         echo " | ";
4769         // Join this table to Contacts

6. Do not see the expected output. Again, as much as I know. See "second_debug_attempt.png" attachment. "test" and "pf_testy" is my test custom data set, with subtype "Any" selected instead of a specific or group of specific, subtypes. None of the Activities with a selective subtype list appear.

7. Hence, problem exists between line 4732 and 4767. I think. Since I don't really know the code, I'm working on this hypothesis for the moment.

8. What's in there? Let's see:
Code: [Select]
4730 /*   echo "entity type  = $entity_type ||\n";
4731  *   echo "subtype = $subType ||\n";
4732  *   echo "GroupID = $groupID ||";
4733  */
4734     switch ($entity_type) {
4735
4736     case "Contact":
4737     case "Individual":
4738     case "Household":
4739     case "Organization":
4740         $jointable = 'civicrm_contact';
4741         break;
4742     case "Event":
4743         $jointable = 'civicrm_event';
4744         break;
4745     case "Participant":
4746         $jointable = 'civicrm_participant';
4747         break;
4748     case "Contribution":
4749         $jointable = 'civicrm_contribution';
4750         break;
4751     case "Activity":
4752         $jointable = 'civicrm_activity';
4753         break;
4754     case "Relationship":
4755         $jointable = 'civicrm_relationship';
4756         break;
4757     case "Membership":
4758         $jointable = 'civicrm_membership';
4759         break;
4760     }
4761
4762     foreach ($tree as $groupkey => $currentgroup) {
4763         if ($groupkey == 'info') { return $data; } // dodges an invalid argument call after all the groups go through // have a look at what $tree outputs to to see why.
4764
4765         $data[$currentgroup['table_name']]['table']['group']  = t('CiviCRM Custom: ') . $currentgroup['title'];
4766
4767         echo $currentgroup['title'];
4768         echo " | ";

9. Lines 4734 through to 4760 are just a switch statement, ignore

10. 4762 uses this $tree var. What is that? ....looks like it's set above the first debug lines:
Code: [Select]
  4722     if ($style == 'Inline') {
   4723         //echo "subtype = $subType --";
   4724         $tree = CRM_Core_BAO_CustomGroup::getTree( $entity_type, CRM_Core_DAO::$_nullObject, null, $groupID, $subType, null);
   4725     }
   4726     else {
   4727         $tree = CRM_Core_BAO_CustomGroup::getTree( $entity_type, CRM_Core_DAO::$_nullObject, null, $groupID);
   4728     }

11. As you can see, I put in another debug statement, and again got the correct answer. I also think that I may have changed the "inline" (all my data types are inline) call to CRM_Core_BAO_CustomGroup::getTree to include either $subType or $groupID - I'm pretty sure one of them was set to null (why??).

12. Time to go looking for CRM_Core_BAO_CustomGroup::getTree (meanwhile thinking, surely the problem is in views, not in BAO....)

13. Find getTree, find if ( $subType ):
Code: [Select]
327         if ( $subType ) {
    328             $subType  = CRM_Core_DAO::VALUE_SEPARATOR . $subType . CRM_Core_DAO::VALUE_SEPARATOR;
    329             $strWhere = "
    330 WHERE civicrm_custom_group.is_active = 1
    331   AND civicrm_custom_field.is_active = 1
    332   AND civicrm_custom_group.extends IN ($in)
    333   AND ( civicrm_custom_group.extends_entity_column_value LIKE '%$subType%'
    334    OR   civicrm_custom_group.extends_entity_column_value IS NULL )
    335 ";
    336             if ( $subName ) {
    337                 $strWhere .= " AND civicrm_custom_group.extends_entity_column_id = {$subName} ";
    338             }
    339         } else {

14. Start to cry at the wormhole opening before me. Go looking for what the hell CRM_Core_DAO::VALUE_SEPARATOR is.

15. Dr Google tells me that there's a post on the Forums about this: http://forum.civicrm.org/index.php?topic=11654.0

16. Read post, comment out line 321 as suggested (except for me that line is 328, see above)

17. Reload Views edit page and whoop! there it is, my custom type sitting there pretty. Haven't done any testing to see what i've broken yet - came here first.  Will be in IRC all day and all tomorrow.
« Last Edit: April 18, 2010, 06:57:51 pm by datakid »

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 18, 2010, 06:38:53 pm
I'm not sure what I should do to actually fix the problem - commenting out a line seems like the wrong way to fix the problem.

I started looking for where VALUE_SEPARATOR is defined....in http://svn.civicrm.org/civicrm/branches/v3.1/CRM/Core/DAO.php there is

Code: [Select]
VALUE_SEPARATOR = "";
Is this the problem?

{edit}
Looks like it's a rendering problem with svn/html - in the file system it looks like:
Code: [Select]
VALUE_SEPARATOR = "^A";
« Last Edit: April 18, 2010, 07:19:49 pm by datakid »

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 18, 2010, 09:52:08 pm
I didn't see the second page of the forum post I listed above (to my shame), dlobo was kind enough to point it out:

http://forum.civicrm.org/index.php/topic,11654.msg51738.html#msg51738

Unfortunately, it didn't work:
Code: [Select]
Database changed
mysql> UPDATE civicrm_custom_group SET extends_entity_column_value = CONCAT(CHAR( 01 ), extends_entity_column_value, CHAR( 01 )) WHERE LOCATE( char( 01 ), extends_entity_column_value ) <= 0;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0  Changed: 0  Warnings: 0


When I uncommented line 328, I could no longer access the data in views...

datakid

  • I post occasionally
  • **
  • Posts: 35
  • Karma: 0
Re: Custom data broken in Views
April 18, 2010, 11:27:54 pm
The following are lines 327 to 346 of CustomGroup.php, which I am trying to mentally step through:

Code: [Select]
        if ( $subType ) {
            //$subType  = CRM_Core_DAO::VALUE_SEPARATOR . $subType . CRM_Core_DAO::VALUE_SEPARATOR;
            $strWhere = "
WHERE civicrm_custom_group.is_active = 1
  AND civicrm_custom_field.is_active = 1
  AND civicrm_custom_group.extends IN ($in)
  AND ( civicrm_custom_group.extends_entity_column_value LIKE '%$subType%'
   OR   civicrm_custom_group.extends_entity_column_value IS NULL )
";
            if ( $subName ) {
                $strWhere .= " AND civicrm_custom_group.extends_entity_column_id = {$subName} ";
            }
        } else {
            $strWhere = "
WHERE civicrm_custom_group.is_active = 1
  AND civicrm_custom_field.is_active = 1
  AND civicrm_custom_group.extends IN ($in)
  AND civicrm_custom_group.extends_entity_column_value IS NULL
";
        }

I was struck by the flow:

1. Do x
2. if  ( $subName ), concatenate more things to $strWhere
3. else overwrite  $strWhere ( note that it's = not .= )

Since $subName is not passed to this function by the calling routine - civicrm\drupal\modules\views\civicrm.views.inc:4722 (roughly 4722) passes NULL in the final field - then the only $strWhere that is used is the final one looking for "civicrm_custom_group.extends_entity_column_value IS NULL" which will be incorrect, since that field is where the $subType is stored..? Am I off track here?

I feel like there should be an extra "AND ( civicrm_custom_group.extends_entity_column_value LIKE '%$subType%' OR"  in the final where clause, or the final where clause should just not exist....??? Of course, I have no idea how many other code points call this function so I'm happy to be wrong :)

Still doesn't explain why simply commenting out line 328...hang on - Why are you adding CRM_Core_DAO::VALUE_SEPARATOR to $subType? From my first attachment above you can see that it already has CRM_Core_DAO::VALUE_SEPARATORs bookending the field....

Pages: 1 [2] 3
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Drupal Modules (Moderator: Donald Lobo) »
  • Custom data broken in Views

This forum was archived on 2017-11-26.