[SOLVED] problem with accents in french

cat1910

Joined: 2005-05-23
Posts: 5
Posted: Mon, 2005-05-23 11:30

Hi,

I've installed the last nightly snapshot of Gallery 2 Beta 3 (23/05/2005) on my website (located on an OVH server)
http://www.catherinevandyk.com/gallery

There is a problem with the accents in the menus, not in the comments I've add in the album or the picture.

For example "propriétaire" appears "Propriétaire".

What can I do ? Thanks a lot for your help, and thanks for that really nice program.

Catherine

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Mon, 2005-05-23 16:23

browse to upgrade/index.php and go up to step 2.. does it report any missing/modified files?
also please fill out the support template.

 
cat1910

Joined: 2005-05-23
Posts: 5
Posted: Mon, 2005-05-23 18:43
mindless wrote:
browse to upgrade/index.php and go up to step 2.. does it report any missing/modified files?
also please fill out the support template.

Just that warning :

Gallery file integrity Warning
- Old files (2)
These files are no longer part of Gallery. They probably won't cause any problems but it is a good idea to remove them to keep your install clean.
modules/core/test/data/test3.gif
modules/core/test/data/localUploadDir/test3.gif


Gallery URL (optional): http://www.catherinevandyk.com/gallery
Gallery version: gallery 2 beta 3 0.9.18
Webserver (with version): Linux web52.300gp.ha.ovh.net 2.4.28-mutu
Datatabase (with version): MySQL 3.23.58
PHP version (eg 4.2.1): PHP Version 4.3.11
phpinfo URL (optional): http://240gp.ovh.net/test.php
Graphics Toolkit(s): exif, gd2
Operating system:
Web browser/version: IE6, Mozilla Firefox 1.0.3

 
cat1910

Joined: 2005-05-23
Posts: 5
Posted: Thu, 2005-05-26 08:13
cat1910 wrote:
mindless wrote:
browse to upgrade/index.php and go up to step 2.. does it report any missing/modified files?
also please fill out the support template.

Just that warning :

Gallery file integrity Warning
- Old files (2)
These files are no longer part of Gallery. They probably won't cause any problems but it is a good idea to remove them to keep your install clean.
modules/core/test/data/test3.gif
modules/core/test/data/localUploadDir/test3.gif


Gallery version = 2.0-beta-3+ core 0.9.18
PHP version = 4.3.11 cgi
Serveur Web = Apache
Base de données = mysql 3.23.58-log
Boîtes à outils = Exif, Gd, ImageMagick, Thumbnail
Système d'exploitation = Linux web53.240.ha.ovh.net 2.4.28-mutu #1 SMP Wed Jan 5 12:22:46 CET 2005 i686
Navigateur Web = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Fri, 2005-05-27 07:28

Same issue here, with the same host (www.ovh.com)

Step 2 says:

System Checks
PHP Version Success
FILE Directive Success
Safe Mode Success
exec() allowed Success
set_time_limit() allowed Success
Memory limit (no limit) Success
File uploads allowed Success
Maximum upload size (2Mb) Success
Maximum POST size (8Mb) Success
Gallery file integrity Warning
+ Modified files (1)

It is a fresh installation.
Note that when I moved from my previous web host to OVH, I had the same problem with my database for drupal which I had to rebuild (the problem is not really the same as here, gallery2 is a fresh install).

Tom

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-05-27 07:32

Tdb, which file is modified?

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Fri, 2005-05-27 14:54

cat1910, I don't know why the characters don't appear correctly.. perhaps try changing some components (php version, isapi instead of cgi, webserver version, or on a different OS..) and see if anything helps..

 
cat1910

Joined: 2005-05-23
Posts: 5
Posted: Fri, 2005-05-27 15:00
mindless wrote:
cat1910, I don't know why the characters don't appear correctly.. perhaps try changing some components (php version, isapi instead of cgi, webserver version, or on a different OS..) and see if anything helps..

:cry: I can't do anything on the webserver, php version, etc...
so I'll have to go back to Gallery 1.
Thanks for your answer !

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-05-27 16:54

mindless, tim_j has the same problem, just for german (deutsch) umlauts.

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Sat, 2005-05-28 07:36
valiant wrote:
Tdb, which file is modified?

Well, I don't know, I was expecting I could click on the "+" sign in front of " Modified files (1)" to see it, but it is not clickable, how can I see it?

Thank,
Tom

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-05-28 09:15

Tdb, deactivated javascript? g2 has some use of javascript, you should activate it.

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Sat, 2005-05-28 10:12

OK, I have now the wrong file:
layouts/classic/po/fr_FR.po

I thought it was a good sign, I have replaced the file on the server with the one from the archive, and now, step 2 shows success for file integrity:
Gallery file integrity => Success

However, the problem remains :(

http://www.vaniche.be/gallery2/main.php

Thanks,
Tom

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sat, 2005-05-28 15:06

Ok, here is a test.php file to try on your server:

<?php
 putenv('LANG=fr_FR');
 putenv('LANGUAGE=fr_FR');
 $locale = setlocale(LC_ALL, 'fr_FR');
 bindtextdomain('modules_core', '/path/to/gallery2/modules/core/locale');
 bind_textdomain_codeset('modules_core', 'UTF-8');
 textdomain('modules_core');
?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html><head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
</head><body>
  Selected locale: <?php var_dump($locale); ?> <br/>
  Translate 'advanced search': <?php print dgettext('modules_core', 'advanced search'); ?>
</body></html>

Set the correct path in the bindtextdomain line. Adjust the 'fr_FR' string in the setlocale call to the right value for your OS if you see "Selected locale: false" when you try the php file.

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Sun, 2005-05-29 10:50

Mindless/codeful, thanks for the outstanding work you do to help me.
I have modified the file according to your recommendations:

bindtextdomain('modules_core', '/home/vaniche/www/gallery2/modules/core/locale');

But I get the following error:
Fatal error: Call to undefined function: bind_textdomain_codeset() in /home/vaniche/www/gallery2/test.php on line 6

From the documentation, it looks like this function was added in php 4.2.0, and ovh uses 4.3.1.
Could the problem be due to this? Is it a function that needs to be enabled at compilation time? Compilation options are visible at this link:
http://60gp.ovh.net/test.php

Cheers,
Tom

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-05-29 11:17

Tdb, thanks for the feedback.
Related php bug reports:
http://bugs.php.net/search.php?search_for=bind_textdomain_codeset&boolean=0&limit=10&order_by=&direction=ASC&cmd=display&status=All&php_os=&phpver=&assign=&author_email=&bug_age=0
see: http://bugs.php.net/bug.php?id=29407
also see (windows):
http://bugs.php.net/bug.php?id=21525&edit=2

but G2 does not depend on bind_textdomain_codeset() for translations. But it could be a sign that other related stuff isn't ok on your system either.

Tdb, can you comment out / remove the line "bind_textdomain_codeset('modules_core', 'UTF-8'); " from mindless' test script and tell us what the result is?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-05-29 11:39

mindless,
i tried the above script, and it wasn't translated. i have fr_FR installed (locale -a) on my system. And yes, I used the correct path to core/locale
this was the output:

 Selected locale: string(5) "fr_FR"
Translate 'advanced search': advanced search

In G2, it works nevertheless perfectly: i get "recherche avancée" in the sidebar when using French.

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Sun, 2005-05-29 12:58

This is the output I get when removing the problematic line:

Selected locale: string(5) "fr_FR"
Translate 'advanced search': advanced search

Thanks,
Tom

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sun, 2005-05-29 15:46

ah, silly me.. that string is localized in the search module, not core.. here's an updated script that also handles not having bind_textdomain_codeset.

<?php
header('Content-Type: text/html; charset=UTF-8');
putenv('LANG=fr_FR');
putenv('LANGUAGE=fr_FR');
$locale = setlocale(LC_ALL, 'fr_FR');
bindtextdomain('modules_search', '/path/to/gallery2/modules/search/locale');
$x = function_exists('bind_textdomain_codeset');
if ($x) bind_textdomain_codeset('modules_search', 'UTF-8');
textdomain('modules_search');
$y = dgettext('modules_search', 'advanced search');
if (!$x) {
require_once('/path/to/gallery2/bootstrap.inc');  require_once('/path/to/gallery2/modules/core/classes/GalleryCoreApi.class');  
  $z = GalleryCoreApi::convertToUtf8($y);
}
?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-s
trict.dtd">
<html><head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
</head><body>
  Selected locale: <?php var_dump($locale); ?> <br/>
  Translate 'advanced search': <?php print $y; ?> <br/>
  <?php if (!$x) print "Convert to UTF-8: $z"; ?>
</body></html>

Note 2 places now to update the path.

edit by valiant: added require_once for bootstrap.inc, now it works for the other cases too.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-05-29 15:59

mindless, ah, that's the problem!!

Now, i pretend to not have bind_textdomain_codeset (just set $x = false):
and this is the rather familiar output:

 Selected locale: string(5) "fr_FR"
Have bind_textdomain_codeset: no
Translate 'advanced search': recherche avancée
 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-05-29 16:02

mindless, and when I don't use bind_textdomain_codeset and comment out the replacement, i get the correct output!
so, is the conclusion that ::convertToUtf8() is not needed?

output in this scenario:

 Selected locale: string(5) "fr_FR"
Have bind_textdomain_codeset: no
Translate 'advanced search': recherche avancée

this was obtained with setting $x = false after the function_exists check and with commenting out
/* if (!$x) {
    
  require_once('/var/www/localhost/htdocs/gallery2/bootstrap.inc');
  require_once('/var/www/localhost/htdocs/gallery2/modules/core/classes/GalleryCoreApi.class');
  $y = GalleryCoreApi::convertToUtf8($y);
}
*/
 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Sun, 2005-05-29 16:05

This is the error message I now get:

Fatal error: Call to a member function on a non-object in /home/vaniche/www/gallery2/modules/core/classes/helpers/GalleryCharsetHelper_simple.class on line 105

Thanks,
Tom

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-05-29 16:16

Tdb, i have changed the code from mindless a few minutes after he has posted it. get the code again and try again, it works for me.

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sun, 2005-05-29 16:24

ya, it seems likely we're doing an unnecessary conversion there. our *.mo files are already UTF-8 after all.. the question is, is there some environment where we do need that conversion, or do we remove it completely?

I modified the code above once more, to show the output before and after convertToUtf8.

Here is the comment in the code about that conversion:

    /*
     * If there is no bind_textdomain_codeset (PHP 4.1.x), the string will be returned
     * from gettext in the default encoding for that language. Here we convert that
     * encoding to the active charset manually.
     */
 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-05-29 16:37

Don't know...
it would be safe if we could detect the encoding of the string and convert only if needed.

Hmm, a use case... what if the string we want to translate isn't in utf8 and the *gettext() function doesn't find a translation?
does it then return non-utf8 again?

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sun, 2005-05-29 17:20

hm, the input strings in G2 should always be english.. so probably that use case won't apply, right?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-05-29 17:33

i have no idea if we call this translation method with english strings that are not utf8.

but if we do and *gettext finds no translation, we could end up with something that is non-utf8.

don't know if we have any english (non-utf8) strings that contain non-utf8 safe characters though :)

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Sun, 2005-05-29 19:42

It looks like it is not really necessary that I give the result anymore, but here it is:

Selected locale: string(5) "fr_FR"
Translate 'advanced search': recherche avancée
Convert to UTF-8: recherche avancée

Thanks,
Tom

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sun, 2005-05-29 19:56

thanks Tdb.. what is your OS/php version?

valiant, I don't think we'll have non-utf8 chars in the input strings.

so, I ran the locale.php test on php 4.1 and found that the convertToUtf8 is required there, as the comment above implies.. now we need to determine under what conditions we skip that conversion..

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Sun, 2005-05-29 20:11

Mindless, it looks like OS/php version used by my web host is Linux 2.4.26 / PHP 4.3.11 :
http://60gp.ovh.net/test.php

Thanks,
Tom

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Sun, 2005-05-29 20:21

Ok, fix committed to cvs.. we're really just guessing that php 4.2+ doesn't need convertToUtf8 of gettext results even if bind_textdomain_charset is missing.. hopefully that works for everyone.

 
Tdb

Joined: 2005-05-27
Posts: 9
Posted: Mon, 2005-05-30 09:57

Yipee :-)
Great job, thanks!

Tom

 
cat1910

Joined: 2005-05-23
Posts: 5
Posted: Tue, 2005-05-31 06:14
mindless wrote:
Ok, fix committed to cvs.. we're really just guessing that php 4.2+ doesn't need convertToUtf8 of gettext results even if bind_textdomain_charset is missing.. hopefully that works for everyone.

Thanks a lot, that works great !

 
xavtek

Joined: 2005-10-22
Posts: 14
Posted: Sat, 2006-03-04 11:14

i was about to up that topic because i have same encoding problems, but it looks like my version of php is just very old (4.1.2).
I can't update it on my production environnement so i'll wait for a major upgrade.
I post about it anyway if it can be usefull or investigated.
TIA

Xavier

 
BigBoss

Joined: 2004-04-14
Posts: 2
Posted: Thu, 2006-04-27 01:19

Ok, I just downloaded the FINAL version, today is the 27thof April 2006, ONE YEAR LATER ! And I have EXACTLY the same problems with the French language !

What do I need to do, install a BETA ?

Please help me !!

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Thu, 2006-04-27 01:38

FAQ: What information is required when I ask for help in the forums?

Probably start a new topic.
Don't know what all the CAPS is about.. there's something wrong with your environment, did you expect us to fix your environment in the past year?

 
xavtek

Joined: 2005-10-22
Posts: 14
Posted: Thu, 2006-04-27 08:40

i do think it's about your environnement too, i can't upgrade to a newer version of php yet so i deal with it as it is. It's no big deal, i switched to english only.