Currency Symbols messed up in CiviCRM 4.4

[Joeri Theelen]

Hi,
I just did a clean installation of CiviCRM 4.4 on Drupal 7.23. I noticed that special symbols such as the Euro sign, are garbled. I tried a clean installation of CiviCRM 4.3.7 and there the special symbols were fine. Same setup. I explicitly changed the collation of the MySQL database to "utf8_general_ci" before doing the installation (in both cases).

Can this be fixed?

Thanks,
Joeri

[Donald Lobo]

any chance you can look at the html and encoding of 4.4 vs 4.3.7 and see what the differentces are

lobo

[Joeri Theelen]

Sure I can. I did a clean installation of CiviCRM 4.3.7 and 4.4 on Drupal 7.23, then went immediately to localization settings, saved the HTML-page and took a screenshot of the page-info in Firefox. See attached file (remove .txt extension !!). I looks like the encoding is the same. The saved pages are missing some styling, but the garbled characters are obviously there in the 4.4 case, not so much in the 4.3.7 case.

I did of course, look into the database script for setting up CiviCRM (e.g. file "civicrm_generated.mysql"), but could not find any differences yet.

Thanks for looking into this.

Joeri

[Coleman Watts]

I just did a clean 4.4 install and the currency symbols all appear normal.
Not sure how to reproduce this. I wonder if it is a windows-specific issue.

[Joeri Theelen]

I did a clean 4.4 install on our production environment, Ubuntu 12.04 Server, same issue. Special symbols are messed up.

I just need one clean 4.4 installation: my workaround is to do a 4.3.7 install + 4.4 upgrade. That works for me.

[Eileen]

Coleman asked me to look at this as he thought it was a Windows issue - I'm not clear if it is since you could replicate it on Ubuntu. It is possible that you still unzipped on windows & the process of unzipping changed the file encoding?

I did do a search for messed up currencies & found a mix of 'good' & 'ungood' currency symbols for BGN (Bulgarian) -see pic

[Joeri Theelen]

Hi, thanks for looking into this. For me, it is not so urgent anymore: I have my 4.4 installed as described in my last message and it is working. But I'd be happy to help you with solving this issue.

I downloaded the tar.gz-archives from Sourceforge, then moved them (SCP) to my Ubuntu server home-folder before unpacking them (tar -xf). I'm not sure it is a Windows issue either. If no one else can repeat this issue, and it is only me, then I suggest you wait until another user comes around with the same issue, and continue from there.

All I can tell is that, in my environment, the 4.3.7 install is correct on Windows and Linux, and the 4.4.0 install isn't. I have done multiple installations, before doing the 4.3.7 + upgrade, which is what I am using now.

Joeri

[Joeri Theelen]

Hi Eileen, I did a pristine and clean 4.4.0 install on Ubuntu 12.04 again, and still got the issue. I have made a dump of the CiviCRM tables in MySQL (structure and data). You can find the SQL dump attached (remove .txt extension). As you can see, the wrong characters are already in the database, so I guess it is a database issue, or an issue with the filling of the database tables during installation. There really should be a difference between the 4.3.7 and 4.4.0 case with regard to installation, but looking at the installation files, I haven't found it yet.

Best,
Joeri

[Michael Loughlin]

I had this problem on a Ubuntu server. I did a clean install on OSX Mavericks, and imported the CiviCRM database into the Ubuntu system to solve the issue. HTH

[Coleman Watts]

So you're saying that installing the same version of Civi (4.4) on Debian and OSX have different results?

[Michael Loughlin]

Sorry, I should have stated that. The install I did on Mavericks showed no problems with currency symbols, unlike the Ubuntu server install. Replacing the Ubuntu CiviCRM database with the one generated on the Mavericks install worked for me. No doubt there are many differences from one environment to the other, php, mysql, Apache config, etc, so I am not able to precisely pinpoint the source of the problem.

[mathieu]

Was the installation directly into another language (en_UK or nl_NL, etc), or was it in the default US English?

The .sql files shipped with CiviCRM, which are imported by the installer, all seem OK.

I could not reproduce it on Debian, and I am wondering if it's specific to Ubuntu, which would be particularly weird.. either that, or the program that uncompresses the .tar.gz archive, although in this case, it's tar.

[Carlo Landmeter]

I have the same issue where it installs with the wrong encoding.
Thinks like currency and state/country all have wrong chars.

This is with latest Drupal and latest CiviCRM on Ubuntu 11.04

[torrance123]

We had this issue on a new civi 4.4 install on debian 6. I had to manually copy across the badly encoded names in the civicrm_currency, civicrm_country and civicrm_state_province tables from another 4.4 version (which had only been upgraded, not wholly installed).

I had a look at the civicrm/sql/civicrm_data.mysql file and could see that all the names were correctly encoded when reading the file in a terminal (terminal locale set to UTF8). Also, the relevant tables (and default database character set) were all set to utf8. So I suspect something is up with either:

- the text encoding civicrm/php is using to read the .mysql files
- or if these are being read by mysql directly, then possibly the mysql client or connection isn't set to utf8

Though these are just guesses at the moment!

[torrance123]

I ran a quick test with the mysql files:

CREATE DATABASE `test` CHARACTER SET = utf8 COLLATE = utf8_general_ci;

And then:

mysql test1 < civicrm/sql/civicrm.sql
mysql test1 < civicrm/sql/civicrm_data.sql

Resulting data is just fine. So the issue lies elsewhere.