Code defect causes failure to create customer field in Chinese

[qizhang]

Hi All,
I'm not sure whether this is the best place to report this problem. Please let me know if I need to open a defect by another channel.

Problem:
This causes any update/insert to customer field and customer field set in Chinese failed by "record already exists" error.

Root cause:
I dived into the code and found the root cause of this problem today.
In the munge method of /Utils/String.php,  there is a line:
$name = preg_replace('/\s+|\W+/', $char, trim($name) );
But if the $name is a Chinese string, the return value of preg_replace always be '_' regardless what the exact value of $name is.

I worked around this problem by add following lines:
$oldName = $name;
$name = preg_replace('/\s+|\W+/', $char, trim($name) );
if ( $name=='_' ) $name = $oldName;

My question here is, is that OK to fill in the "name" column of civicrm_custome_field with Chinese characters? Because by doing this, the value of "name" column may be exactly same with "label" (i.e,. with Chinese characters). Not sure whether this can introduce any other problems?

One of my idea to fix it is use the BASE64 code of Chinese characters as the "name". But I am not a PHP programmer so I don't know the best way to implement it.

Thanks

[xavier]

Hi,

you don't have the pb with  other fields? eg groups?

munge is a way to get a "computer friendly" name. Evidently it doesn't work with chinese charsets. Bypassing that might cause problems, eg in the api or ajax interfaces or mysql I think.

Would be good it you ask developers in your county how they solve that (probably a common problem)

X+

[Donald Lobo]

in "latin" based languages the effect of that reg-exp is to replace non-alpha numeric (and hence potentially non-printable) characters with a '_'

If you can come up with a regular expression that has the same effect for all character sets, we could potentially replace the regexp we currently use

lobo