Firefox PHP

Problem with german and french letters

Posted by Kajover 
Problem with german and french letters
February 24, 2006 01:37PM
Hey,

Does anybody know how to make Phorum cope with letters, like ä, ö, ü, ê. à, á, etc...

Phorum is running in German and Wordpress is UTF-8. Should be a problem in the header.tpl, because Phorum worked fine til the integration.
Integration: Modified header.tpl and footer.tpl

[PHP]<?php
/* Don't remove this line. */
require('../wp-blog-header.php');
?>
<?php get_header(); ?>


<div align="{forumalign}">
<div class="PDDiv">
{IF notice_all}
<div class="PhorumNotificationArea PhorumNavBlock">
{IF PRIVATE_MESSAGES->new}<a class="PhorumNavLink" href="{PRIVATE_MESSAGES->inbox_url}">{LANG->NewPrivateMessages}</a><br />{/IF}
{IF notice_messages}<a class="PhorumNavLink" href="{notice_messages_url}">{LANG->UnapprovedMessagesLong}</a><br />{/IF}
{IF notice_users}<a class="PhorumNavLink" href="{notice_users_url}">{LANG->UnapprovedUsersLong}</a><br />{/IF}
{IF notice_groups}<a class="PhorumNavLink" href="{notice_groups_url}">{LANG->UnapprovedGroupMembers}</a><br />{/IF}
</div>
{/IF}
<span class="PhorumTitleText PhorumLargeFont">
{IF NAME}<a href="{URL->TOP}">{NAME}</a>&nbsp;:&nbsp;{/IF}{TITLE}</span>
{IF DESCRIPTION}<div class="PhorumFloatingText">{DESCRIPTION}</div>{/IF}

<style type="text/css">

/* Element level classes */

td, th
{
color: Black;
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
}

img
{
border-width: 0px;
vertical-align: middle;
}

a
{
color: #000099;
text-decoration: none;
}
a:active
{
color: #FF6600;
text-decoration: none;
}
a:visited
{
color: #000099;
text-decoration: none;
}

a:hover
{
color: #FF6600;
}

input[type=text], input[type=password], input[type=file], select
{
/* border: 1px solid #808080; */

background-color: transparent;
color: Black;
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;

vertical-align: middle;

}

textarea
{
background-color: White;
color: Black;
font-size: 12px;
font-family: Lucida Console, Andale Mono, Courier New, Courier;
}

input[type=submit]
{
border: 1px dotted #808080;
background-color: #ff6c00;
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
vertical-align: middle;
}

input
{
vertical-align: middle;
}

/* Standard classes for use in any page */
/* PhorumDesignDiv - a div for keeping the forum-size size */
.PDDiv
{
width: 720px;
padding: 20px;
text-align: left;
}
/* new class for layouting the submit-buttons in IE too */
.PhorumSubmit {
border: 1px dotted #808080;
color: Black;
background-color: #ff6c00;
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
vertical-align: middle;
}

.PhorumTitleText
{
float: left;
padding: 6px;
}

.PhorumStdBlock
{
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
background-color: White;
border-left: 0px solid #808080;
border-right: 0px solid #808080;
/* width: 100%; */
padding: 3px;
}

.PhorumStdBlockHeader
{
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
background-color: #ff6c00;
/* width: 100%; */
border-left: 0px;
border-right: 0px;
border-top: 1px solid #808080;
padding: 3px;
}

.PhorumHeaderText
{
font-weight: bold;
}

.PhorumNavBlock
{
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
border-top: 1px solid #808080;
border-right: 0px;
border-bottom: 1px solid #808080;
border-left: 0px;
margin-top: 1px;
margin-bottom: 1px;
/* width: 100%; */
background-color: transparent;
padding: 2px 3px 2px 3px;
}

.PhorumNavHeading
{
font-weight: bold;
}

A.PhorumNavLink
{
color: #000000;
text-decoration: none;
font-weight: normal;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
font-size: 12px;
border-style: solid;
border-color: #ff6c00;
border-width: 1px;
padding: 0px 4px 0px 4px;
}

A.PhorumNavLink:hover
{
background-color: #FFFFFF;
font-weight: normal;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
font-size: 12px;
border-style: solid;
border-color: #808080;
border-width: 1px;
color: #FF6600;
}

.PhorumFloatingText
{
padding: 10px;
}

.PhorumHeadingLeft
{
padding-left: 3px;
font-weight: bold;
}

.PhorumUserError
{
padding: 10px;
text-align: center;
color: Red;
font-size: 16px;
font-family: Trebuchet MS,Verdana, Arial, Helvetica, sans-serif;
font-weight: bold;
}

.PhorumNewFlag
{
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
font-size: 10px;
font-weight: bold;
color: #CC0000;
}

.PhorumNotificationArea
{
float: right;
border-style: dotted;
border-color: #808080;
border-width: 1px;
}

/* PSUEDO Table classes */
/* In addition to these, each file that uses them will have a */
/* column with a style property to set its right margin */

.PhorumColumnFloatXSmall
{
float: right;
width: 75px;
}

.PhorumColumnFloatSmall
{
float: right;
width: 100px;
}

.PhorumColumnFloatMedium
{
float: right;
width: 150px;
}

.PhorumColumnFloatLarge
{
float: right;
width: 200px;
}

.PhorumColumnFloatXLarge
{
float: right;
width: 400px;
}

.PhorumRowBlock
{
background-color: White;
border-bottom: 1px solid #F2F2F2;
padding: 5px 0px 0px 0px;
}

.PhorumRowBlockAlt
{
background-color: #EEEEEE;
border-bottom: 1px solid #F2F2F2;
padding: 5px 0px 0px 0px;
}

/************/


/* All that is left of the tables */

.PhorumStdTable
{
border-style: solid;
border-color: #808080;
border-width: 0px;
width: 100%;
}

.PhorumTableHeader
{
background-color: #ff6c00;
border-top: 1px solid #808080;
border-right: 0px;
border-bottom: 1px solid #808080;
border-left: 1px;
color: #000000;
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
font-weight: bold;
padding: 3px;
}

.PhorumTableRow
{
background-color: White;
border-bottom-style: solid;
border-bottom-color: #F2F2F2;
border-bottom-width: 1px;
color: Black;
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
height: 35px;
padding: 3px;
}

.PhorumTableRowAlt
{
background-color: #EEEEEE;
border-bottom-style: solid;
border-bottom-color: #F2F2F2;
border-bottom-width: 1px;
color: #000000;
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
height: 35px;
padding: 3px;
}

table.PhorumFormTable td
{
height: 26px;
}

/**********************/


/* Read Page specifics */

.PhorumReadMessageBlock
{
margin-bottom: 5px;
}

.PhorumReadBodySubject
{
color: Black;
font-size: 16px;
font-family: Trebuchet MS,Verdana, Arial, Helvetica, sans-serif;
font-weight: bold;
padding-left: 3px;
}

.PhorumReadBodyHead
{
padding-left: 5px;
}

.PhorumReadBodyText
{
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
padding: 5px;
}

.PhorumReadNavBlock
{
font-size: 12px;
font-family: Verdana, Lucida Sans Unicode, Lucida Grande, Arial;
border-left: 0px solid #808080;
border-right: 0px solid #808080;
border-bottom: 1px solid #808080;
/* width: 100%; */
background-color: #ff6c00;
padding: 2px 3px 2px 3px;
}

/********************/

/* List page specifics */

.PhorumListSubText
{
color: #707070;
font-size: 10px;
font-family: Arial, Helvetica, sans-serif;
}

.PhorumListPageLink
{
color: #707070;
font-size: 10px;
font-family: Arial, Helvetica, sans-serif;
}

.PhorumListSubjPrefix
{
font-weight: bold;
}

.PhorumListModLink, .PhorumListModLink a
{
color: #707070;
font-size: 10px;
font-family: Arial, Helvetica, sans-serif;
}
/********************/

/* Override classes - Must stay at the end */

.PhorumNarrowBlock
{
width: 600px;
}

.PhorumSmallFont
{
font-size: 11px;
}

.PhorumLargeFont
{
color: Black;
font-size: 16px;
font-family: Trebuchet MS,Verdana, Arial, Helvetica, sans-serif;
font-weight: bold;
}


.PhorumFooterPlug
{
margin-top: 10px;
font-size: 10px;
font-family: Arial, Helvetica, sans-serif;
}</style>[/PHP]
Re: Problem with german and french letters
February 24, 2006 02:13PM
which phorum-version do you run?


Thomas Seifert
Re: Problem with german and french letters
February 24, 2006 02:42PM
version 5.0.21
Re: Problem with german and french letters
February 24, 2006 04:50PM
you need to add a meta-tag with the charset. don't know if your wp-header adds this.


Thomas Seifert
Re: Problem with german and french letters
February 24, 2006 08:53PM
Where do I add this and how would it look like exactly?
Re: Problem with german and french letters
February 25, 2006 05:22AM
Normally, you would set $PHORUM["DATA"]['LANG_META']=""; in your language file to contain something like this:
  <meta http-equiv="Content-Type" content="text/html;iso-8859-1" >
But you're now using the WP headers in your templates. That is okay, but apparently the WP headers do send an UTF-8 declaration. That will at least make your language file work incorrect, because the character encoding that is used in there is iso-8859-1. That could be fixed by creating an UTF-8 encoded version of the language file.

What I'm wondering is if you have the problem for the language file and already posted messages or also for new posts. My guess is that new posts show the accents correctly, because they are sent and displayed as UTF-8. In that case, having an UTF-8 language file would be the way to go, because there's no good way to mix up character encodings within a single page.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problem with german and french letters
February 25, 2006 05:36AM
And where or how do I get an UTF-8 language file in German? By the way, I have got the same "problem" when I switched the language of Phorum to the default one, English.

New Topicsare shown correctly.... Only the menu of phorum and my first topic are displayed incorrectly.



Edited 2 time(s). Last edit at 02/25/2006 05:40AM by Kajover.
Re: Problem with german and french letters
February 25, 2006 08:07AM
Switching to whatever language won't fix this. The problem is that your WP templates tell the browser to expect UTF-8 encoded data, but the data in the language file and the postings that were done when using Phorum without WP templates is encoded using ISO-8859-1.

I'll try to explain a bit what encoding does:

Characters are represented by one or more bytes of data. There is no universal way of storing an é or ë. Instead, applications have to agree on the way that these characters are represented in the data. In encoding method X, an ë might be represented by number 123, while in encoding method Y, the same character might be represented by 234. So what encoding basically is, is a ruleset for how to represent characters in the data. Now if you write a text using encoding method Y containing an ë, then 234 will be used for it in the data. Now if you feed that exact data to an application using method X, then you will get the character that uses 234 from that encoding. To be able to use data that was produced with encoding method X in encoding method Y, you have to do encoding conversion, which will take care of changing 234 -> 123, so encoding X will show ë too.

The old topics and the language file were all created using ISO-8859-1 ("encoding X"), but you're now telling the browser to using UTF-8 ("encoding Y"). Therefore, those are not shown correctly. If you post new messages, the browser will encode the data using UTF-8 (you told the browser to do so) and it will therefore be stored in UTF-8. This is why new postings do show up correctly. If you would be able to change the encoding somehow, the old posts and language would be readable, but the new postings would be garbage.

I hope this makes a bit more clear what encoding is about and why it can be such a pain in the b*tt.

Now ("Na endlich!" I can hear you thinking) for a possible solution. I wrote a little script to convert the german.php file to UTF-8. PHP luckily has utf8_encode() as a function which will conveniently turn ISO-8859-1 into UTF-8. The result can be found in german-utf8.zip. This file contains the orignal german.php, the conversion script and the resulting UTF-8 version german-UTF8.php. Please try if the UTF8 file from the zip-file works for you.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce



Edited 1 time(s). Last edit at 02/22/2007 07:45AM by mmakaay.
Re: Problem with german and french letters
February 25, 2006 10:35AM
Woahh... I did not thought that there are still friendly people out there. THX a lot!!!! (I am at a loss of words!)

Everything is now readable, except of the title.
Forum : Quart&#65533;te &#65533; T&#65533;te &#65533;ffentlicher Bereich
Should be Quartête À Tête Öffentlicher Bereich...

-> I've uploaded the 3 files into the langfolder. Run the converting... deleted the files except of German-UTF8.php, which I assigned as the standardlanguage for each forum. <- Mistake?

If the problem is not found I can write he tilte, like this: Oeffentlicher Bereich. Wouldn't be a problem^^


But THX anyway, I'm dumbfounded!
Re: Problem with german and french letters
February 25, 2006 12:24PM
Hehe, no problem.

You did not have to do anything with the conversion script, because the zip already contains the UTF8 german file. It was mainly packed for people that want to see how it was done. But it won't hurt either if you run this code yourself, like you did.

Why the title shows up like this, I can't explain right now. Maybe it's because in the admin interface (where you set the forum title) UTF-8 is not used. Pushing that interface into UTF-8 would work I think. If you like to try, you can add the following line somehwere to the start of the admin.php script.
  header("Content-type: text/html; chartype=UTF-8");
If this works, then after editing the title of the forum, it could be correct in the front end pages.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problem with german and french letters
February 25, 2006 01:28PM
I did not use umlauts for the title so the problem is solved. Thx again^.
Re: Problem with german and french letters
May 31, 2006 03:24AM
Hi,
I had a similar issue with character encoding. My issue was slightly different because the settings of my MySQL database was set to use UTF8_GENERAL_CI for the default collation of all tables.
This means that all characters are encoded in the DB as UTF8 and that was preventing PHORUM from displaying the characters correclty because the default MySQL PHP connector is not using UTF8.
Since I used the converter from Phorum 3 to Phorum 5 on this database, all characters from previous posts were now UTF8 encoded and I was seeing everything wrong.

To fix that, I edited the files mysql.php and mysqli.php in include/db so it connects using UTF8 encoding.

before (mysql.php):
function phorum_db_mysql_connect(){
$PHORUM = $GLOBALS["PHORUM"];

static $conn;
if (empty($conn)){
$conn = mysql_connect($PHORUM["DBCONFIG"]["server"], $PHORUM["DBCONFIG"]["user"], $PHORUM["DBCONFIG"]["password"], true);
mysql_select_db($PHORUM["DBCONFIG"]["name"], $conn);
}
return $conn;
}

after (mysql.php):
function phorum_db_mysql_connect(){
$PHORUM = $GLOBALS["PHORUM"];

static $conn;
if (empty($conn)){
$conn = mysql_connect($PHORUM["DBCONFIG"]["server"], $PHORUM["DBCONFIG"]["user"], $PHORUM["DBCONFIG"]["password"], true);
mysql_select_db($PHORUM["DBCONFIG"]["name"], $conn);
// set the mode to UTF8
if($conn!=false) {
mysql_query("SET NAMES 'utf8'", $conn);
mysql_query("SET CHARACTER SET utf8", $conn);
}
}
return $conn;
}

before (mysqli.php):

function phorum_db_mysqli_connect(){
$PHORUM = $GLOBALS["PHORUM"];

static $conn;
if (empty($conn)){
$conn = mysqli_connect($PHORUM["DBCONFIG"]["server"], $PHORUM["DBCONFIG"]["user"], $PHORUM["DBCONFIG"]["password"], $PHORUM["DBCONFIG"]["name"]);
}
return $conn;
}


after (mysqli.php):
function phorum_db_mysqli_connect(){
$PHORUM = $GLOBALS["PHORUM"];

static $conn;
if (empty($conn)){
$conn = mysqli_connect($PHORUM["DBCONFIG"]["server"], $PHORUM["DBCONFIG"]["user"], $PHORUM["DBCONFIG"]["password"], $PHORUM["DBCONFIG"]["name"]);
// set the mode to UTF8
if($conn!=false) {
mysqli_query( $conn, "SET NAMES 'utf8'");
mysqli_query( $conn, "SET CHARACTER SET utf8");
}
}
return $conn;
}


I hope this helps some others and I think the PHORUM dev team should maybe move to a fully compliant UTF8 mode folowing this method.
Fabrice.
Re: Problem with german and french letters
May 31, 2006 03:39AM
guess what this forum runs in?
yeah, utf-8.

guess if there were hacks needed?
yeah, none ;).

the point is to set the database server correctly in utf-8 mode, its not phorum that needs to be changed.


Thomas Seifert
Re: Problem with german and french letters
May 31, 2006 03:48AM
well, ok but then could you explain what I did wrong in this:
I was using phorum 3, I installed phorum 5.1.12 and I ran the phorum3to5convert.php wich worked fine.
Then I went to the newly installed phorum 5 to check if everything was imported correclty and I saw a LOT of ? instead of the accentuated characters.
I checked the values in the DB using phpmyadmin and they show up correcty there so I did the fix I talked about in the previous post to fix that.

I would be happy to use PHORUM as it has been implemented but then I need to know what I should do instead of my fix, could you please help?
thanks.
Re: Problem with german and french letters
June 01, 2006 06:01PM
Just set the database and tables to utf-8, i.e. use the my.cnf settings too. everything is returned as utf-8 then.
set utf-8 in the language-file and you should be set, like we are here.


Thomas Seifert
Re: Problem with german and french letters
June 02, 2006 07:01AM
Hello,
Thanks for your answer.
I checked and my DB was alredy setup as you say; this is the output from phpmyadmin:
MySQL - 5.0.18
* Protocol version: 10
* Server: localhost via TCP/IP
* MySQL charset: UTF-8 Unicode (utf8)
* MySQL connection collation: utf8_unicode_ci

I changed the language file (include/lang/french.php) to utf-8:
$PHORUM['DATA']['CHARSET'] = "utf-8";

I also checked that my page charset was set to be utf-8 because I use the portable version of Phorum to integrate it in my application:
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">

The phorum menus (coming from the language file) are displayed correctly but not the posts coming from the migration from phorum 3.
I tried to submit a new post with specials characters and everything is fine so I guess it is something done during the migration.

Should this be considered a bug then?

Thanks,
Fabrice.
Re: Problem with german and french letters
June 02, 2006 07:37AM
No, it's not a bug. It's a database storage problem. In the old tables, data was probably stored as something like iso-8859-1, but the encoding for the new forum is UTF-8. New posts get sent and stored as UTF-8, but the old posts are still in iso-8859-1, hence they swow up wrong. In able to make this work, the data that is stored in mysql has to be converted to UTF-8 before use.

Now, unfortunately, I'm no encoding wizard with MySQL, so I wouldn't know if there is a simple way to pull this off. Maybe others can tell what to do here.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problem with german and french letters
June 02, 2006 08:49AM
So, pretty much, I could write a script that reads each row from mysql in iso-8859-1, converts each row in UTF8 using utf8_convert and then update the row back in UTF-8...I will try that but I guess the migration script should take care of that.
To me, it is not a bug in PHorum but it looks like one in the migration script.

On the other hand, I do not understand why doing this right after the mysqli_connect makes things work correctly:
mysqli_query( $conn, "SET NAMES 'utf8'");
mysqli_query( $conn, "SET CHARACTER SET utf8");

Thanks.
Re: Problem with german and french letters
June 02, 2006 08:51AM
the phorum3 to phorum5 conversion isn't meant to do any charset conversion. it only takes the data from one data-structure and puts in the new one.


Thomas Seifert
Re: Problem with german and french letters
July 20, 2006 04:28PM
Quote
fdouteaud
On the other hand, I do not understand why doing this right after the mysqli_connect makes things work correctly:
mysqli_query( $conn, "SET NAMES 'utf8'");
mysqli_query( $conn, "SET CHARACTER SET utf8");.

This is because MySQL does transcode the content from the table/column charset to the declared connection charset.
Since MySQL 4.1 you can mix charsets as much as you want as long as the data stored use the encoding declared to Mysql, AND you understand the MySQL encoding paradigm (which took me some hours).

BTW, off topic, I came here in search for the best forum to fit my needs and utf-8 is a requirement. Another one is that i18n use gettext (for best integration with my own languages files). Looks like this is not the case, can anyone confirm?



Edited 1 time(s). Last edit at 07/20/2006 04:34PM by Max_B.
Re: Problem with german and french letters
July 20, 2006 04:59PM
Confirmed. Phorum uses its own language system for doing the translations.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problem with german and french letters
February 28, 2007 06:40PM
Good evening,

a few days ago my webhosting provider has changed something in PHP (I do not know what exactly) and there started the problem with some special letters in Czech language (diacritics), specially with č š ž ť. I have changed the file mysql.php as it is recommended above (to Latin2) and there is only one improvement - in new answer messages, letters are OK, but: names of topics, users, old messages (all messages until today) and also changed messages are still including ? ? ? (question marks) instead of those four letters.

I can see, that changing that mysql.php helped a little, but I do not know, what to do next, so that all phorum is OK with those letters.

Important info: i changed coding to Latin2 everywhere. But database is by default in cp1250_general_ci. And since my webhosting provider has changed sthing in PHP, those special czech letters are also in database displayd as ??? (question marks).

Please, if anybody can help... We have been using Phorum for more than one year without any problem and now this...

Tom
Re: Problem with german and french letters
February 28, 2007 07:12PM
Have you informed with your provider for what they changed exactly? Maybe the database has been converted to UTF-8, but it's hard to tell without knowing the background of the changes. You could try changing the character encoding in your browser to find out in which charset the data is correctly on the screen (in Firefox it's under View -> Character encoding).

Changing the charset on a live system is not without peril. New messages that are posted, will be posted in the new charset. This might result in having different character encoded messages in the database (so when switching back to the old charset, the new messages might not be readable anymore).


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problem with german and french letters
March 01, 2007 04:25AM
OK, I am going to ask the provider, what exactly has been changed.

I tried to change encoding in Firefox and the best result is giving ISO-8859-2.

Yes, I did not realized that it might be very dangerous if we managed to display the old messages correctly, that the new messages will not be OK anymore...

It´s rather difficult for me, because I don´t understand, why when I look directly into the database, there are question marks instead of š ž and ť (special czech letters).

As soon as I get info from the provider, what has been changed, I will post it here. If you have any idea, I will appreciate it. Thanks a lot.
Re: Problem with german and french letters
March 01, 2007 08:45AM
So, my webhosting provider has sent me e-mail, that they changed only the version of mysql client, so that this change did not affect database server.

They also told me that there had to be question marks (instead of those few letters) in the database already before the change of mysql client (i can neither agree nor disagree, because i did not study the messages in the database). But even if there were question marks before this change, in the phorum it was displayd correctly (letters).

Very strange is, that insterting that code into include/db/mysql.php affected only answer posts (when i post an answer to somebodys post), not new posts, not names of new topics, not user names... So it looks like this file mysql.php were used only for answers... but I did not find any other file where I should insert that code:

mysql_query("SET NAMES 'latin2'", $conn);
mysql_query("SET CHARACTER SET latin2", $conn);

Are there any other files to change by insterting this code? It is really strange for me, that only answer posts are correct and nothing else...



Edited 1 time(s). Last edit at 03/01/2007 08:46AM by Thomass.
Re: Problem with german and french letters
December 01, 2007 04:23PM
hi everybody,
I decided to revive this thread because the problems that I experience are related to this thread. It seems that this problem is quite common for those who use certain special letters in some languages, for instance in my case - Lithuanian language - we use these letters ą č ę ė į š ų ū ž. By the way, one has the same problem using virtually every message board system or blog system (the problem is fixed in newest versions of Wordpress, which I'll explain later).

Now the main problem is the impossibility to properly restore Phorum from the mysql backup: Lithuanian letters ą č ę ė į š ų ū ž are not restored - you see only some crazy signs like Ä? instead of Lithuanian letters. Moreover, backup files doesn't show Lithuanian letters properly. But before restoring everything was fine in the Phorum - all the Lithuanian letters were OK. Besides, it's important to note that when you have a correct Lithuanian phorum and you check mysql database via phpMyAdmin, you cannot see any Lithuanian letters properly (you see something like 'ęėšėįšųšįų" there).

So in sum, Lithuanian letters function perfectly all right until one tries to restore phorum from the backup. At the same time, everything seem to work properly in utf-8: phorum runs in utf-8, webbrowser character encoding is in utf-8, MySQL charset: UTF-8 Unicode (utf8).

This is very annoying especially when one has to use his backup files quite often. And it is difficult to understand what's going on. It seems like something goes wrong with utf-8 when phorum connects mysql or something. But I am no expert in this field, so I don't know for sure.

Now it is interesting that Wordpress somehow solved this problem: in earlier versions there was the same problem but now it was fixed: althouh I don't know the internals of wordpress, now in config.php one can write "define('DB_CHARSET', 'utf8');" and the everything is fine: backup files show Lithuanian letters, then one can restore all the messages properly, and mysql database show Lithuanian letters (via phpMyAdmin).

I have heard that some people used some fixes (like in the message above):
mysql_query("SET NAMES 'utf8'", $conn);
mysql_query("SET CHARACTER SET utf8", $conn);

mysqli_query( $conn, "SET NAMES 'utf8'");
mysqli_query( $conn, "SET CHARACTER SET utf8");

But since in Phorum 5.2.2 files mysql.php and mysqli.php have changed, I do not know where to put these lines. and is it ok to insert them? (fdouteaud doesn't seem to visit this site, so I can't ask). maybe there is any efficient way to solve this problem?

I would appreciate any suggestions. Thank you.
Re: Problem with german and french letters
December 01, 2007 06:05PM
What procedure do you use for creating backups? This sounds like the backup is broken already, since after creating a backup dump, it does not contain the correct characters. Is that because they are really really wrong in the data or are you possibly using a non-UTF-8 viewer for looking at the dump?


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problem with german and french letters
December 01, 2007 06:56PM
Quote
mmakaay
What procedure do you use for creating backups? This sounds like the backup is broken already, since after creating a backup dump, it does not contain the correct characters. Is that because they are really really wrong in the data or are you possibly using a non-UTF-8 viewer for looking at the dump?
I use the standard way to create dump of database (save as file as SQL) through phpMyAdmin, but in fact I had used other methods as well (as some integrated methods of backup creation in some blog systems, etc.) And the problem is not related to viewers, I have used hundreds of viewers :) - the result is always the same: Lithuanian and other special letters are not shown properly and (what is most important) Lithuanian letters are not restored.

BTW as I said, the problem can be seen immediately when I open the database in phpMyAdmin and don't see correct Lithuanian letters (although at the same time Phorum messages are shown OK before deleting phorum and restoring it from the backup). As I tried to explain in the example with Wordpress, when they starting using "define('DB_CHARSET', 'utf8');" everything got right: Notepad shows the backup dump correctly, phpMyAdmin also shows it correctly, the restoration goes all right, etc.

This is all a mysterious issue. I've even heard that there some bugs in mysql but anyway all this is possible to solve through message boards and blogs systems and I hope through Phorum fixes.

Anyway, it seems that nobody in the virtual world knows a straightforward answer. So I am desperate to try above mentioned fix or smth similar:
mysql_query("SET NAMES 'utf8'", $conn);
mysql_query("SET CHARACTER SET utf8", $conn);

mysqli_query( $conn, "SET NAMES 'utf8'");
mysqli_query( $conn, "SET CHARACTER SET utf8");

Maybe you know where I could try to insert it? :)
Re: Problem with german and french letters
December 01, 2007 07:33PM
I don't think it's the level where this should be fixed (before dumping things are okay, after restoring they are broken, meaning that your data changed), but if you want to try this, then hack this into include/db/mysq/mysql.php or include/db/mysql/mysqli.php, depending on the PHP extension that you are using.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problem with german and french letters
December 01, 2007 09:11PM
no, it didn't work, the form of crazy characters has changed but it got even worse - Phorum now show no Lithuanian letters at all :) the problem seem to be somewhere deeper...
Sorry, only registered users may post in this forum.

Click here to login