Author Topic: Export of custom data - missing correct header and data issue? (Read 4144 times)

Raphy · December 08, 2009, 07:12:46 pm

When I try to export various Custom Data Fields, the csv returns header with "custom_23" and with no data in it.

On previous occasions the header of the column would be "old_intranet_id_23"

I know there is data associated with the contact record and you can see the correct data and headers directly in the mysql database tables.

Any ideas on what is going wrong, what I should try next? ( and how to fix it ) ? :-)

Kurund Jalmi · December 09, 2009, 07:47:33 am

Can you replicate this on http://drupal.demo.civicrm.org and attach sample file.

Thanks
Kurund

matth3wh · April 23, 2010, 12:15:04 am

This appears to be an ongoing issue for us... Strangely the problem went away and has come back again :-(
We have done several updates across the time of this issue - 3.0.x and 3.1.x ... currently on 3.1.3

Sorry Kurund - it is not easy to replicate this issue on the demo platform mainly because we didn't notice a clear list of repeatable steps to cause the problem... :-(

Can you suggest what we could look for directly in the database which might give a clue ?

matth3wh · April 23, 2010, 12:46:42 am

Just did a search on a particular surname (individuals) then export contacts then select fields for export

Generates a stack of these errors at the top of the page:
# warning: array_merge() [function.array-merge]: Argument #1 is not an array in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 590.
# warning: Invalid argument supplied for foreach() in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 606.

Followed by the normal screen Select Fields to Export (step 3 or 3)...

I then put in some export mapping values and success - the CSV has the correct column header...

I click the DONE button and I get the following errors:

# warning: array_merge() [function.array-merge]: Argument #1 is not an array in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 590.
# warning: Invalid argument supplied for foreach() in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 606.
# warning: array_merge() [function.array-merge]: Argument #1 is not an array in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 590.
# warning: Invalid argument supplied for foreach() in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 606

matth3wh · April 23, 2010, 12:58:56 am

Just tried Find Member -> surname -> particular membership

Then Export Members for selected records

Then Export PRIMARY fields

Still get the problem... :-(

custom_20 custom_21 custom_88

Full list of PRIMARY fields in the resulting CSV

Internal Contact ID Contact Type Contact Subtype Do Not Email Do Not Phone Do Not Mail Do Not Sms Do Not Trade No Bulk Emails (User Opt Out) Legal Identifier External Identifier (match to contact) Sort Name Display Name Nick Name Legal Name Website Image Url Preferred Communication Method Preferred Mail Format Source of Contact Data First Name Middle Name Last Name Job Title Birth Date Is Deceased Deceased Date Mail to Household ID Household Name Organization Name Sic Code Unique ID (OpenID) Current Employer ID Addressee Email Greeting Postal Greeting Gender Individual Prefix Individual Suffix Current Employer location_type Street Address Street Number Street Number Suffix Street Name Street Unit Supplemental Address 1 Supplemental Address 2 City Postal Code Suffix Postal Code Geo Code 1 Geo Code 2 Address Name County State Country Phone Email On Hold Use for Bulk Mail im_provider IM Screen Name Openid World Region Phone Type Im Service Provider Membership Type Test Is Pay Later Member Since Membership Start Date Membership Expiration Date Source Membership Status Membership ID Primary Member ID custom_20 custom_21 custom_88 Total Amount Contribution Status Received Date Payment Instrument Transaction ID

NOTE: Select fields for export results in more of these errors...
# warning: array_merge() [function.array-merge]: Argument #1 is not an array in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 590.
# warning: Invalid argument supplied for foreach() in /var/www/sites/all/modules/civicrm/CRM/Core/BAO/Mapping.php on line 606.

Donald Lobo · April 23, 2010, 07:45:19 am

i would start off with debugging those warnings and figuring out why the array_merge is giving a warning. so adding some debug statements above it will give you some clues. you might want to compare it with an export that does not produce a warning

lobo

matth3wh · April 26, 2010, 04:43:41 pm

I think this may be the cause of the problems ... A most likely associated strange glitch / bug I've noticed now is that under Admin -> Configure -> Import/Export Mappings (civicrm/admin/mapping?reset=1&action=browse) I get the errors as mentioned in previous posts array_merge() line 590 and invalid arg for each line 606... plus the first two entries in this section are blank! The name Description Mapping Type are all blank... You cannot delete them and you can only edit their name and description fields.

On our install Mapping Type for real export/import entries is - {ts escape="sql"}Export Membership{/ts} where as on the Demo site it is just Export Membership

Also I'm puzzled why export primary fields (for the Memberships) is showing the custom_xx columns. Is that by design ?

matth3wh · April 26, 2010, 05:29:01 pm

Wahoo...! No... it is not fixed but I've reproduced it on the drupal.demo.civicrm.org site ;-)

Steps to reproduce the problem on a new site / demo site:

1. Admin -> Customize -> Custom Data -> New Group of Custom Fields (button)
2. Create a new group of custom data Used for Memberships... e.g.
  Group Title   Enabled?   Used For   Type
   MailingDetails    Yes     Memberships   Any

3. add custom fields...
e.g. Field Label    Data Type   Field Type   Order   Req?   Enabled?
   Copies    Integer    Text 1 Yes
   DeliveryMethod   Alphanumeric   Select 2 Yes
   MembershipDescription   Note    TextArea 3 Yes

4. add a new Membership for an individual...
e.g. (on the demo site) Alexey Kaminskij -> Memberships -> New Membership

5. add entries to the custom data fields for the new Membership
e.g. Membership Type -> General
   MailingDetails -> Copies -> 1
   -> DeliveryMethod -> by email
   -> MembershipDescription -> This is a test

6. Save that

7. (Now try to Export Membership data...) In the menu -> Memberships... -> Find Members
8. Search
9. Select Records (e.g. All 31 records)
10. Change the drop-down menu (more actions) to Export Members -> Go
11. Choose Export PRIMARY fields or Select fields for export.... (pick the new custom data fields)
12. examine the CSV and the column headers for our new Membership custom data will be custom_5 custom_6 custom_7
   instead of copies ; DeliveryMethod ; MmbershipDescription

Results:

1. membership related custom data is exported as custom_X ... instead of the correct column headers... (where X is some number in the order that the custom data was created)
2. admin -> configure -> Import/Export Mappings gets one or more blank entries (blank name description and Mapping Type)

Possible Results:
1. this hasn't happened on the demo site but in our example we get BAO/Mapping.php errors... array_merge() line 590 and invalid argument for each line 606...

Dave Greenberg · August 04, 2010, 10:09:24 am

This issue was reported on issue tracker and marked resolved as of 3.1.5:
http://issues.civicrm.org/jira/browse/CRM-6161

However a recent post has reported recurrence of the behavior with custom fields in Contribution (and possibly Contact) exports.

http://forum.civicrm.org/index.php/topic,14938.0.html

awjrichards · August 04, 2010, 11:30:24 am

I actually managed to resolve this by clearing all of the CiviCRM caches (using the drush script). I'm hoping that's all there is to it. Thanks!

CiviCRM Community Forums (archive)

News:

Author Topic: Export of custom data - missing correct header and data issue? (Read 4144 times)

Raphy

Export of custom data - missing correct header and data issue?

Kurund Jalmi

Re: Export of custom data - missing correct header and data issue?

matth3wh

Re: Export of custom data - missing correct header and data issue?

matth3wh

Re: Export of custom data - errors

matth3wh

Re: Export - find member custom field column headings incorrect and no data

Donald Lobo

Re: Export of custom data - missing correct header and data issue?

matth3wh

Re: Export of custom data - missing correct header and data issue?

matth3wh

Re: Export Memberships Custom Data - missing correct header and data issue?

Dave Greenberg

Re: Export of custom data - missing correct header and data issue?

awjrichards

Re: Export of custom data - missing correct header and data issue?