CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • Using UFT-16 to properly display special characters in Windows
Pages: [1]

Author Topic: Using UFT-16 to properly display special characters in Windows  (Read 2176 times)

JoopSJ

  • I post occasionally
  • **
  • Posts: 80
  • Karma: 2
Using UFT-16 to properly display special characters in Windows
November 26, 2009, 05:39:13 am
A CiviCRM contact-search export will not display special characters (é ñ etc.) properly if the file is opened by Excel or another windows based apllication. This problem does not occur using Open Office.

After trying a lot of things I can only solve this by changing the encoding to UTF-16. It looks like this is the only encoding that supports MultiByte charachters in Excel.

I addapted the encoding in CRM/Core/Report/Excel to UTF-16 by adding this line just prior the echo to file lines:

Code: [Select]
$out = mb_convert_encoding($out, "UTF-16", "UTF-8");

I know not all installs have access to mb_convert_encoding, but for those who have it, this might be a good solution.
If it is, is it feasible to have en admin-option to select the export encoding to UTF-16? If this option is set, the encoding will be set as described above.

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Using UFT-16 to properly display special characters in Windows
November 26, 2009, 10:03:42 am
That's not that excel and the gang don't understand utf8, just that they try to make it as painful as possible to use it (ie. by default they go all windows encoding, itself a wicked iso latin 1 encoding).

Anyway, save the .csv as .txt, then go on excel, then file open the txt. then excel let you choose the char encoding (unicode utf-8 or something like that).

Et voila...

X+

-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

JoopSJ

  • I post occasionally
  • **
  • Posts: 80
  • Karma: 2
Re: Using UFT-16 to properly display special characters in Windows
November 26, 2009, 11:39:40 am
Well that's the point.
For my end-users "saving as and selecting the proper encoding"  is not an option.
Also we have a use-case where after saving the export, another application (Excel-like) will take care of the processing of the file.
The current CRM-application (the one CiviCRM is replacing) can do this without handchanging the file encoding by the end-user.

So I'll have to change this in the code (I promised them improvements  ;D)

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Using UFT-16 to properly display special characters in Windows
November 27, 2009, 02:43:25 am
Yeap, fix the stuff that isn't broken to work around the one that is broken but you can't fix... now the feeling ;)


Anyway, have you tested on openoffice ? Does it import too utf-16 ?

If i'm right, the files are twice larger (for mostly english names), right ?

To make it easier, might be good to add a test if the mb_ function exists, so that's spits utf8 normally, and convert to 16 if mb exists ?

X+
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • Using UFT-16 to properly display special characters in Windows

This forum was archived on 2017-11-26.