Problem with search option and Polish letters

[siristru]

I've changed Localization in to Polish so it is my default language now.
Many entries in my CiviCRM contains Polish letters like "ł,ą,ś,ć". When I try to search contacts that contains those letters I've get "no matches" and Polish letters are turning into: "stanis�aw".

Could enybody help me to overcome this problem? Search using Polish letters is crucial in my CRM because most of our names contains it.

[Michał Mach]

Hey,

CiviCRM uses UTF-8, do Polish characters should be working fine. Can you check your database encoding, as well as Apache encoding?

In the database, you can check it using following command:
show variables where variable_name like 'character%';

In Apache, you can just hit Ctrl-I when on CiviCRM page and see what's the "Encoding" set to.

Pozdrowienia, :-)
m

[Piotr Szotkowski]

You can also verify the issue on our demo server – if it happens there, it’s a bug, if not, it’s a problem with your configuration.

Również pozdrawiam. :)

[siristru]

Thanks guys for your advices.

In database I have:

MySQL charset:  UTF-8 Unicode (utf8)
MySQL connection collation: utf8_unicode_ci

character_set_client             utf8
character_set_connection    utf8
character_set_database    utf8
character_set_filesystem    binary
character_set_results            utf8
character_set_server            latin2
character_set_system            utf8

In my case CiviCRM and Joomla 1.5.14 uses the same database with the same encoding.

So it looks fine. In table civicrm_contact entries also look fine - "Stanisław" is without any "?".

I've checked on your demo site - searching works perfect there.

I don't know how to check Apache encoding. I don't see access to Apache from my administrator panel at admin.nazwa.pl

Today I tried to use search again straight from "Search contacts" (not as before "Build search"). This time just displayed "No results". But for sure I have a contact Stanisław as a name...

Kłaniam się serdecznie

[Michał Mach]

Hey,

To check the encoding that Apache uses, you can just hit Ctrl-I in your browser (or choose Tools >> Page Info, at least on my Mac) when viewing CiviCRM page and see what's the "Encoding" set to. However, I would be surprised if Nazwa wasn't using UTF-8 for their webservers.

Did you import your contacts from Excel? If so, can your try saving the same file in Open Office and importing it again?

Thx,
m

[siristru]

I use FireFox 3.5.3 on Windows XP. So when I hit Ctrl + I I get opened panel with Favorites.

I imported those entries from OpenOffice.org - converting them into *csv with encoding UTF8 (encoding checked in kED and Notepad).

But even if I create new contact manually via "Create contact", CiviCRM doesn't see it when using search.

I have character_set_server latin2 and the res is in UTF8... should it be like that?

[Michał Mach]

On Joomla demo, where it works for you, only character_set_system is set to utf8 and it works, so I don't think there is a problem here (we set table collaction and connection encoding on our side):


mysql> show variables where variable_name like 'character%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | latin1                     |
| character_set_connection | latin1                     |
| character_set_database   | latin1                     |
| character_set_filesystem | binary                     |
| character_set_results    | latin1                     |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+

Maybe you can invoke page information by right clicking on CiviCRM page and choosing Page Info?

Thx,
m

[siristru]

Ok, right click shows page info:

Date: Mon, 19 Oct 2009 06:38:22 GMT
Server: Apache/2
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
Expires: Mon, 1 Jan 2001 00:00:00 GMT
Last-Modified: Mon, 19 Oct 2009 06:38:23 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Keep-Alive: timeout=2, max=50
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8

I've instaled CiviCRM 3.0 on xampp and problem is the same. Installation is automatic so problem should lies somewhere in configuration

[Michał Mach]

How about "Encoding" information (placed above what you pasted, in Page Info window)?

Also, does any encoding problem happen in Joomla as well?

Yet another theory - can you check what kind of encoding do you have set in your Firefox (View -> Character encoding in my FF)? It should be UTF-8.

In general, I don't think at this stage that the problem lies in the database, most problably it's somewhere "on the way" between db and the browser - especially that from what I understand, you're experiencing this problem only in search. (am I right?)

Thx,
m

[siristru]

In FF i have UTF-8 encoding.

Yes, problem appears only in search.

Maybe I'll try to use search in other browser?

[siristru]

So I've tried on other browser (Opera 9.64) with automatic encoding detection. Problem is the same.
Even when I changed in FF encoding to Automatic -> Universal, nothing changed.

But I noticed interesting thing. When using quick search field at CiviCRM top panel, typing "Stanisław" system gives a suggestions showing all contacts with "Stanisław" line. But pressing enter again gives "No matches found".

[siristru]

Anybody could help me? In this stage CiviCRM (great extension indeed) is useless

[Piotr Szotkowski]

Is there a chance we can get access to your install? It would be the easiest way to debug this problem.

[siristru]

Piotr, yes there is a chance that I could give you access to my install. I trust you So please tell me where I should send access data. Should I create an account for you?
By the way, please contact me via this mail: siristru @ go2.pl