language encoding in php data files

dmizer

Joined: 2008-10-08
Posts: 40
Posted: Wed, 2008-10-08 01:21

Please fill out the following form and take a look at the linked pages. The more information you supply, the better we can help you. Remove this FAQ information from your post before submitting.


G2 URL (optional): offline
G2 version: 2.2.4
G1 URL (optional):
G1 version: v1.5.7
G1 character set: en-us ISO-8859-1,utf-8
PHP memory limit: 32M
--
PHP version (e.g. 5.1.6): 5.2.4
PHPInfo Link (see FAQ): Will provide this if absolutely necessary.
Webserver (e.g. Apache 1.3.33): lighttpd/1.4.19
Database (e.g. MySql 5.0.32): MySql 5.0.51a-3ubuntu5.1
Activated toolkits (e.g. NetPbm, GD): NetPBM, ImageMagick
Operating system (e.g. Linux): Linux Ubuntu 8.04
Browser (e.g. Firefox 2.0): Firefox 3.0.3

More relevant information:
Multibyte (japanese) regex support enabled
_SERVER["HTTP_ACCEPT_LANGUAGE"] en-us,en;q=0.5
_SERVER["HTTP_ACCEPT_CHARSET"] ISO-8859-1,utf-8;q=0.7,*;q=0.7

I am attempting to convert my gallery from g1 to g2. Most of this has gone quite well; however, some of my comments have Japaense characters in them, and they are not showing up correctly in the description fields of g2. With a bit of research, I discovered that in my g1 album.dat files, the Japanese characters are showing up as raw code like so "福岡".

Since I have a great number of pictures and comments with a load of Japanese, I would like to know if there's a way to convert the Japanese in the dat files so it will import to g2 correctly.

Although there is probably a forum post about this somewhere, I was unable to find it.

Edit: I have read this FAQ and added this line to my config.php

$gallery->setConfig('systemCharset', 'UTF-8');

When I browse to the page, the browser is now selecting UTF-8 for the character encoding, but the Japanese is still not displaying.

Edit:
I "fixed" this by employing some friends and going through every file and updating them by hand. :(
 
dmizer

Joined: 2008-10-08
Posts: 40
Posted: Thu, 2008-10-09 12:39

Shameless bump. I have had zero luck with this.

I have tried manually converting the g1 php files to utf8 with the following command:
recode UTF-8 foo.dat

I have made sure that the entire database was set to utf8.

I have followed the directions here: http://dev.mysql.com/doc/refman/5.0/en/charset-conversion.html to make sure the entire database is correctly encoded.

Why will my Japanese not display correctly in Gallery 2, but it displays correctly in Gallery 1?
 
dmizer

Joined: 2008-10-08
Posts: 40
Posted: Fri, 2008-10-10 00:01

I've discovered that something must be happening during the upload to the mysql database. When I import an album via the G2 admin panel, the Japanese displays correctly. After import, the Japanese does not display correctly.

Is there any more information I can provide to help solve this vexing problem?
 
dmizer

Joined: 2008-10-08
Posts: 40
Posted: Fri, 2009-11-06 18:53

Okay ...

Here's the old gallery link (with how the text should appear):
New gallery link (with messed up text):

Key text is:

Quote:
First stop: Fukuoka (福岡). Spent the first couple days of this week long holiday here.

I've seriously given this my best shot here. Nothing I've found anywhere seems to address this particular problem.

Help ... please?
 
dmizer

Joined: 2008-10-08
Posts: 40
Posted: Fri, 2008-10-10 16:04

Found the problem, don't know how to fix it.

The text in the mysql database is fine, and reads 福

But the page source shows that the "&" is '&' for syntactically valid xhtml. In order for the text to be read correctly, the & needs to be html literal, not xhtml. Is there a way I can get around this? This is happening somewhere in the gallery 2 code, not in the database.