Author Topic: Code defect causes failure to create customer field in Chinese  (Read 1890 times)

Offline qizhang

  • I’m new here
  • *
  • Posts: 16
  • Karma: 0
  • CiviCRM version: 4
  • CMS version: 7
  • MySQL version: 5.1.41
  • PHP version: 5.2
Code defect causes failure to create customer field in Chinese
« on: November 12, 2011, 09:05:39 am »
Hi All,
I'm not sure whether this is the best place to report this problem. Please let me know if I need to open a defect by another channel.

Problem:
This causes any update/insert to customer field and customer field set in Chinese failed by "record already exists" error.

Root cause:
I dived into the code and found the root cause of this problem today.
In the munge method of /Utils/String.php,  there is a line:
$name = preg_replace('/\s+|\W+/', $char, trim($name) );
But if the $name is a Chinese string, the return value of preg_replace always be '_' regardless what the exact value of $name is.

I worked around this problem by add following lines:
$oldName = $name;
$name = preg_replace('/\s+|\W+/', $char, trim($name) );
if ( $name=='_' ) $name = $oldName;

My question here is, is that OK to fill in the "name" column of civicrm_custome_field with Chinese characters? Because by doing this, the value of "name" column may be exactly same with "label" (i.e,. with Chinese characters). Not sure whether this can introduce any other problems?

One of my idea to fix it is use the BASE64 code of Chinese characters as the "name". But I am not a PHP programmer so I don't know the best way to implement it.

Thanks
« Last Edit: November 12, 2011, 09:16:56 am by qizhang »

Offline xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Code defect causes failure to create customer field in Chinese
« Reply #1 on: November 12, 2011, 09:33:34 am »
Hi,

you don't have the pb with  other fields? eg groups?

munge is a way to get a "computer friendly" name. Evidently it doesn't work with chinese charsets. Bypassing that might cause problems, eg in the api or ajax interfaces or mysql I think.

Would be good it you ask developers in your county how they solve that (probably a common problem)

X+

Offline Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Code defect causes failure to create customer field in Chinese
« Reply #2 on: November 12, 2011, 02:56:48 pm »

in "latin" based languages the effect of that reg-exp is to replace non-alpha numeric (and hence potentially non-printable) characters with a '_'

If you can come up with a regular expression that has the same effect for all character sets, we could potentially replace the regexp we currently use

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions