Author Topic: Hebrew (Read 13116 times)

FredJones · July 31, 2007, 12:03:07 pm

1. Israel is not in the country list for v 1.8 as far as I can see.

2. The symbol for ILS doesn't appear correctly in my browser.

3. Truth is that I have a CiviCRM with lots of Hebrew data, and it works, but as I am trying to upgrade to 1.8 the Hebrew is failing. NOt sure why yet...

Piotr Szotkowski · August 01, 2007, 12:51:35 am

Quote from: FredJones on July 31, 2007, 12:03:07 pm

1. Israel is not in the country list for v 1.8 as far as I can see.

Hm, it’s right there in my install. Are you sure you don’t have the country list restricted in
Administer CiviCRM → Global Settings → Localization?

Quote from: FredJones on July 31, 2007, 12:03:07 pm

2. The symbol for ILS doesn't appear correctly in my browser.

Does your Drupal site serves its content in UTF-8? If you have Firefox, this is easily seen with the View Page Info right-click menu entry (or after pressing ctrl+i).

Quote from: FredJones on July 31, 2007, 12:03:07 pm

3. Truth is that I have a CiviCRM with lots of Hebrew data, and it works, but as I am trying to upgrade to 1.8 the Hebrew is failing. Not sure why yet...

This is very strange. What exactly is failing?

FredJones · August 01, 2007, 01:10:12 am

Quote from: Piotr Szotkowski on August 01, 2007, 12:51:35 am

Hm, it’s right there in my install. Are you sure you don’t have the country list restricted in
Administer CiviCRM ? Global Settings ? Localization?

My fault for not mentioning that that is actually the place where I do NOT see it.

Even a view page source and search for "Israel" proves me right.

Quote from: Piotr Szotkowski

Does your Drupal site serves its content in UTF-8? If you have Firefox, this is easily seen with the View Page Info right-click menu entry (or after pressing ctrl+i).

Encoding is indeed UTF-8. I see this instead of Hebrew: ¢×¢×¢ (if some of those are like signs for cents, then that is what I see in my browser)

Quote from: Piotr Szotkowski

This is very strange. What exactly is failing?

As above--I get junk chars instead of Hebrew.

Same for the ILS currency symbol.

PS: I am on IRC if you want to chat.

Piotr Szotkowski · August 01, 2007, 02:04:53 am

Quote from: FredJones on August 01, 2007, 01:10:12 am

Quote from: Piotr Szotkowski on August 01, 2007, 12:51:35 am
Hm, it’s right there in my install. Are you sure you don’t have the country list restricted in
Administer CiviCRM ? Global Settings ? Localization?
My fault for not mentioning that that is actually the place where I do NOT see it.

Do you have it in the civicrm_country table? It should have an id of 1106.

Quote from: FredJones on August 01, 2007, 01:10:12 am

Quote from: Piotr Szotkowski
Does your Drupal site serves its content in UTF-8? If you have Firefox, this is easily seen with the View Page Info right-click menu entry (or after pressing ctrl+i).

Encoding is indeed UTF-8. I see this instead of Hebrew: ¢×¢×¢ (if some of those are like signs for cents, then that is what I see in my browser)

Hm. Do you see a right-facing arrows in my forum post above (in the Administer CiviCRM → Global Settings… string)? You seem to have quoted them as question marks, so maybe it’s your browser…

Quote from: FredJones on August 01, 2007, 01:10:12 am

PS: I am on IRC if you want to chat.

I’m Chastell on #civicrm on freenode.

FredJones · August 01, 2007, 05:46:09 am

Quote from: Piotr Szotkowski on August 01, 2007, 02:04:53 am

Do you have it in the civicrm_country table? It should have an id of 1106.

Yes, but it's in unicode. Now that I see it in the DB, I do see that it actually DOES show up in my list, but last, b/c it's nonsense characters.

Quote from: Piotr Szotkowski on August 01, 2007, 02:04:53 am

Hm. Do you see a right-facing arrows in my forum post above (in the Administer CiviCRM ? Global Settings… string)? You seem to have quoted them as question marks, so maybe it’s your browser…

Yes I do.

See here: http://tinyurl.com/3yyyp7 for a full description of the issue. We have at that location these files:

1.jpg - This shot shows the same user record when viewed in the online version (Drupal 4.7 and CiviCRM 1.4) and in my local test version with Drupal 5.2 and CiviCRM 1.8. Online it's Hebrew and local it's nonsense
2.jpg - Shows that Page Info appears to be the same for both pages
3.jpg - Shows the same user record when viewed in SQLyog, my MySQL client. Both online and local DBs are shown and would appear to be the same data
1.htm - Source code of v1.4 page saved as Unicode using Notepad--all garbage, oddly enough.
2.htm - Source code of v1.8 page saved normally using Notepad. ODDLY enough, NOW I see Hebrew!
3.htm - I opened 1.htm in Nodepad and did Save As. Now it looks correct.

I am a bit confused myself now.

Michał Mach · August 02, 2007, 05:13:56 am

Hey,

I'll take over you support request, since Piotr took off for vacation.

My first suspicion is: I assume you've added localised name for Hebrew yourself, did your db client had UTF8 set for connection to db server? Sometimes it's not enough to have db encoding set, you also need to set client connection's encoding. For more details see here: http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html

Thx,
m

FredJones · August 02, 2007, 07:02:47 am

Yes, I have Hebrew on my machine.

As far as the client connect, I am fairly certain I never did that on the 1.4 site, not to mention that this:

Quote

2.htm - Source code of v1.8 page saved normally using Notepad. ODDLY enough, NOW I see Hebrew!

would seem to indicate that the correct data is actually on the page, it is just not being rendered correctly by the browser.

I think anyhow--please clarify if this is wrong.

Michał Mach · August 02, 2007, 07:30:45 am

Correct me if I'm wrong, but here is how I envision the process that you came through:
1. you took the dump of database from your online version with 1.4 (what was client connection encoding for this dump? Do you see Hebrew characters in database dump when you open it with notepad and with writepad /I assume you're on windows/?)
2. you installed this dump on your local machine and made a schema upgrade (what was client connection encoding for importing database?) to run with 1.8 code

Let me know if I imagine above process correctly.

Now - if above is right, and after seeing headers for both instances, I only see one potential problem: somewhere along the way database dump's encoding was modified and I don't see any other reasons it was done than wrong client connections during dumping/importing. I'm also thinking it might be some kind of an issue with mysql on different platforms (am I right that one instance runs on *nix and the other on Windows?), but let's verify above guess first. If you want to eliminate second possibility though, please make a clean database install of CiviCRM and make the same change to country table that you made on online server - than we will know if platform difference is potential cause or not.

If the process I described above is not right, please let me know how did you end up with two instances running the same data (but displaying characters differently).

Thx,
m

Michał Mach · August 02, 2007, 07:34:54 am

Skip the Windows/Unix question, I took another look at page info screen dumps and see the answer there: that they are running on different platforms.

m

FredJones · August 02, 2007, 07:52:07 am

> Let me know if I imagine above process correctly.

Something like that. In short, while you were typing this message, I also realized (due to your previous one) that this is the issue. I checked and it does appear that I am using an operating system known as Windows. This is a little-known and somewhat buggy OS that causes headaches, but is very expensive and so it looks good. Anyway, it actually appears that the culprit is UltraEdit. I see now that if I open the CiviCRM .mysql files directly in UltraEdit, the UTF characters are garbage but in NotePad, the characters are correct. Now I made a new CiviCRM DB (without my data) and the UTF characters are correct.

> Now - if above is right, and after seeing headers for both instances, I only see one potential problem: somewhere along the way database dump's encoding was modified and I don't see any other reasons it was done than wrong client connections during dumping/importing.

Reason seems to be my local editor actually.

> I'm also thinking it might be some kind of an issue with mysql on different platforms (am I right that one instance runs on *nix and the other on Windows?),

Nope, MySQL on both handles UTF fine.

> If you want to eliminate second possibility though, please make a clean database install of CiviCRM and make the same change to country table that you made on online server - than we will know if platform difference is potential cause or not.

Oddly enough I made no change whatsoever to the country table. What I wrote above is what I found. Not sure why.

At any rate, this problem seems to be fixed and I will now try to get my data into 1.8. One small problem is that the 1.4 install uses MySQL Version 4.0.27-standard and the 1.8 uses MySQL 5.0. What I will try is to:

1. Rebuild 1.4 into test DB MySQL 5.0 via SQL dump.
2. Run update SQLs for MySQL 4.1 for 1.4 -> ... -> 1.8
3. Insert this data into my 1.8 MySQL 5.0 DB

Thank you very much for your assistance.

Michał Mach · August 02, 2007, 10:16:23 am

Quote from: FredJones on August 02, 2007, 07:52:07 am

> Let me know if I imagine above process correctly.
Something like that. In short, while you were typing this message, I also realized (due to your previous one) that this is the issue. I checked and it does appear that I am using an operating system known as Windows. This is a little-known and somewhat buggy OS that causes headaches, but is very expensive and so it looks good.

Ubuntu is free and looks good as well.

Quote from: FredJones on August 02, 2007, 07:52:07 am

> Now - if above is right, and after seeing headers for both instances, I only see one potential problem: somewhere along the way database dump's encoding was modified and I don't see any other reasons it was done than wrong client connections during dumping/importing.

Reason seems to be my local editor actually.

If you didn't save after opening that file in UltraEdit, it shouldn't really matter. If was guessing again, I assume you edited the file and changed Latin alphabet Israel country name to it's Hebrew equivalent. Now my suggestion would be to make this change through SQL "update" statement, should be more reliable. But you seem to have the problem under control, so I won't make any more guesses. :-)

Quote from: FredJones on August 02, 2007, 07:52:07 am

At any rate, this problem seems to be fixed and I will now try to get my data into 1.8. One small problem is that the 1.4 install uses MySQL Version 4.0.27-standard and the 1.8 uses MySQL 5.0. What I will try is to:

1. Rebuild 1.4 into test DB MySQL 5.0 via SQL dump.
2. Run update SQLs for MySQL 4.1 for 1.4 -> ... -> 1.8
3. Insert this data into my 1.8 MySQL 5.0 DB

This should work.

Thx,
m

FredJones · August 02, 2007, 11:32:50 am

Quote from: Micha? Mach on August 02, 2007, 10:16:23 am

Ubuntu is free and looks good as well.

Have that on a bootable CD.

Quote from: Micha? Mach on August 02, 2007, 10:16:23 am

This should work.

Well I tried it using SQLyog to dump to disk and then rebuild and it failed.

I see that SQLyog 6.0 purports to use UTF-8 internally, but perhaps this is the problem, and so now I am retrying with phpMyAdmin. I do see now that while the Hebrew shows up as garbage characters using either interface, it is DIFFERENT garbage (for some characters) so it seems something may be there.

If this also fails then I will try setting the Import "Character set of the file: " to "Hebrew" instead of "utf8"

But you see the last letter of your first name in my post? I just copied and pasted the reply from the browser to UltraEdit, to edit it, and the CP'd it back. UTF character seems to have gotten lost...

Thanks.

Michał Mach · August 02, 2007, 11:59:55 am

Quote from: FredJones on August 02, 2007, 11:32:50 am

Quote from: Micha? Mach on August 02, 2007, 10:16:23 am
Ubuntu is free and looks good as well.

Have that on a bootable CD.

Transfer to your hard drive as soon as possible, install Beryl/Compiz and start being happy.

Quote from: FredJones on August 02, 2007, 11:32:50 am

Quote from: Micha? Mach on August 02, 2007, 10:16:23 am
This should work.

Well I tried it using SQLyog to dump to disk and then rebuild and it failed.

I see that SQLyog 6.0 purports to use UTF-8 internally, but perhaps this is the problem, and so now I am retrying with phpMyAdmin. I do see now that while the Hebrew shows up as garbage characters using either interface, it is DIFFERENT garbage (for some characters) so it seems something may be there.

If this also fails then I will try setting the Import "Character set of the file: " to "Hebrew" instead of "utf8"

But you see the last letter of your first name in my post? I just copied and pasted the reply from the browser to UltraEdit, to edit it, and the CP'd it back. UTF character seems to have gotten lost...

Don't see the character, but that might be an UltraEdit thing. When it comes to SQLyog and phpMyAdmin on Windows, I cannot help you too much, don't use any of those tools (well, phpMyAdmin a little bit). Actually, I would be curious to see how our forums handle different encodings - if you could sign you next post in Hebrew alphabet, I could see how does it work. Not only different alphabet, but also different text orientation - I really want to see how SMF handles it.

Converting the file to Hebrew might break things, so be careful with that. CiviCRM & Drupal will be setting page and data encoding to UTF-8, so there might be some discrepancy here.

What I would recommend is getting the dump from the server, not touching it with any editors, but importing it to new database and than communicating with it through mysql command line. It's a bit of a challenge at the beginning, but once you get to know it, you will at least have all the settings (especially encodings) under your control.

Good luck,
m

FredJones · August 02, 2007, 12:13:38 pm

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am

Transfer to your hard drive as soon as possible, install Beryl/Compiz and start being happy.

Have thought about switching to Linux for years but I have no time to relearn my tools. A few I use heavily, such as UltraEdit. Some UE users actually do install Win emulators or whatever JUST to run UE. Aside from that, I can't live without "Allnetic Working Time Tracker" That's about it however at this point.

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am

Don't see the character, but that might be an UltraEdit thing. When it comes to SQLyog and phpMyAdmin on Windows, I cannot help you too much, don't use any of those tools (well, phpMyAdmin a little bit). Actually, I would be curious to see how our forums handle different encodings - if you could sign you next post in Hebrew alphabet, I could see how does it work. Not only different alphabet, but also different text orientation - I really want to see how SMF handles it.

If you can see this line:

שלום

in another language, then we got it.

Should be a letter like an o on the left and a letter like a u but with an extra line in the middle, on the right side. If so, then you are seeing "Shalom."

I anyhow see it fine.

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am

Converting the file to Hebrew might break things, so be careful with that. CiviCRM & Drupal will be setting page and data encoding to UTF-8, so there might be some discrepancy here.

OK

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am

What I would recommend is getting the dump from the server, not touching it with any editors, but importing it to new database and than communicating with it through mysql command line. It's a bit of a challenge at the beginning, but once you get to know it, you will at least have all the settings (especially encodings) under your control.

Shared hosting doesn't provide mysql command line.

Michał Mach · August 02, 2007, 12:25:38 pm

שלום

(is it the right word to use instead of "Hey" at the beginning of the message?) :-)

Quote from: FredJones on August 02, 2007, 12:13:38 pm

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am
Transfer to your hard drive as soon as possible, install Beryl/Compiz and start being happy.

Have thought about switching to Linux for years but I have no time to relearn my tools. A few I use heavily, such as UltraEdit. Some UE users actually do install Win emulators or whatever JUST to run UE. Aside from that, I can't live without "Allnetic Working Time Tracker" That's about it however at this point.

We're getting offtopic here, so I won't press you anymore.

I remember not being able to imagine switching to Linux - and it lasted only until I switched.

I still have Windows on my workstation, but didn't boot with it for many, many months now.

Quote from: FredJones on August 02, 2007, 12:13:38 pm

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am
Don't see the character, but that might be an UltraEdit thing. When it comes to SQLyog and phpMyAdmin on Windows, I cannot help you too much, don't use any of those tools (well, phpMyAdmin a little bit). Actually, I would be curious to see how our forums handle different encodings - if you could sign you next post in Hebrew alphabet, I could see how does it work. Not only different alphabet, but also different text orientation - I really want to see how SMF handles it.

If you can see this line:

שלום
in another language, then we got it.

Should be a letter like an o on the left and a letter like a u but with an extra line in the middle, on the right side. If so, then you are seeing "Shalom."

I anyhow see it fine.

Looks perfectly fine to me as well.

I guess I would want it to be aligned to the right side, but perhaps I'm expecting too much. When I get to the line with the word, my browser behaves totally fine. Which means - we have R2L enabled forum. Cool.

Quote from: FredJones on August 02, 2007, 12:13:38 pm

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am
Converting the file to Hebrew might break things, so be careful with that. CiviCRM & Drupal will be setting page and data encoding to UTF-8, so there might be some discrepancy here.

OK

Quote from: Micha? Mach on August 02, 2007, 11:59:55 am
What I would recommend is getting the dump from the server, not touching it with any editors, but importing it to new database and than communicating with it through mysql command line. It's a bit of a challenge at the beginning, but once you get to know it, you will at least have all the settings (especially encodings) under your control.

Shared hosting doesn't provide mysql command line.

But you can connect remotely from your desktop - just give mysql command line host and port parameters and it should attach to remote server without a problem. Technically, it's almost the same way as your CiviCRM install is connecting to the database, so they shouldn't be blocking it.

However, as said before, there is a learning curve in the beginning which requires some time, so you might want to start learning it after you fix your db encoding problems.

Thx,
m

CiviCRM Community Forums (archive)

News:

Author Topic: Hebrew (Read 13116 times)

FredJones

Hebrew

Piotr Szotkowski

Re: Hebrew

FredJones

Re: Hebrew

Piotr Szotkowski

Re: Hebrew

FredJones

Re: Hebrew

Michał Mach

Re: Hebrew

FredJones

Re: Hebrew

Michał Mach

Re: Hebrew

Michał Mach

Re: Hebrew

FredJones

Re: Hebrew

Michał Mach

Re: Hebrew

FredJones

Re: Hebrew

Michał Mach

Re: Hebrew

FredJones

Re: Hebrew

Michał Mach

Re: Hebrew