Module: External Phorummail

OK, based on my preliminary testing, email with "Content-Type: text/html; charset=ISO-8859-1" is properly imported. Obviously this is based on only a few tests and was not thorough by any means but it does look good.

Thank you once again Thomas. :)

Jim

Nginx Forum

Thank you for your feedback.
I was already under the impression that the parser library is far better in handling different charsets.

---

Thomas Seifert

Quote
Thomas Seifert
Good to know that there's someone else using that code too :)

Other than me, right? :-)

I'm under the assumption I can go ahead and try this out, even with the gloomy 'latin1' cloud hovering over my head. I will as soon as I get the time.

I had been collecting emails imported blank in pop3phorummail for you (they all had attachments), but I'll test them all against this script instead.

- Bob

Well, using, not planning to use :D.

You should be able to try it even alongside pop3phorummail as long as the both are going to different forums and email accounts.

your setup with charsets is a bit weird - as far as I remember your forums itself were shown as utf8, right? (check the charset shown in the browser when visiting the forums)
so it might be that you are storing utf8 data in a latin1 database.

---

Thomas Seifert



Edited 1 time(s). Last edit at 09/05/2009 03:39AM by Thomas Seifert.

Quote
Thomas Seifert
your setup with charsets is a bit weird - as far as I remember your forums itself were shown as utf8, right? (check the charset shown in the browser when visiting the forums)
so it might be that you are storing utf8 data in a latin1 database.

You mean this?

Yeah, that shows utf8.
But your database was/is latin1.
So you might be storing utf8 data in a latin1 database.
not a safe thing to do but might work as long as your users mostly use iso/ascii characters.

---

Thomas Seifert

Quote
Thomas Seifert
Yeah, that shows utf8.
But your database was/is latin1.
So you might be storing utf8 data in a latin1 database.
not a safe thing to do but might work as long as your users mostly use iso/ascii characters.

Some imported mail will have some goofy looking charachters in the message bodies.

But in the areas where my members can see and participate, my Phorum is working flawlessly.

However, is there a way to correct the situation? I know Maurice said it's not an easy thing to do to convert the database. But would it be easier if I deleted the forum with all the imported mail? The one you saw was really just to test the script, but it does make up the bulk of the messages right now. I wouldn't care about deleting that one, and keeping the forums that I actually have users participating in.

Or, can a new database (utf8) be created, and then the information from the one database be imported into the new one?

I don't know anything about databases so I am just throwing out theories. Of course, if the best thing to do is leave what's well enough alone, that's cool too. I'll just have to get used to this latin1 weirdo stigma following me around :-)

- Bob

Well, converting the database tables to UTF8 is easy enough. There's MySQL commands for changing the character set. But it's the PHP serialized data inside the database that might be broken due to such change. If there are no special characters in the serialized fields, then conversion would not hurt a bit, but special characters would break the data badly.

I wrote a test script a while back to see if I could restore the broken serialized data (this needs to be done before accessing the Phorum in the browser to be fully sure that all data is handled). You could give this one a go, after converting the database tables to UTF8: [secretsauce.phorum.org]

Deleting the forum with imported mail won't help, since the complete database is in a different character set.

---

Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce

Well, that script even handles the mysql commands to do the conversion. I was searching for it but couldn't find it again before :).
So, I would suggest to give it a go. At least in a copy of the data. And change the charset in the db-config to utf8 afterwards.

---

Thomas Seifert

Quote
Thomas Seifert
Well, that script even handles the mysql commands to do the conversion. I was searching for it but couldn't find it again before :). So, I would suggest to give it a go. At least in a copy of the data. And change the charset in the db-config to utf8 afterwards.

Well okay. I'm gonna hunt for a way make a backup copy of this database later before I do. I'm off to the county fair today :-)

I'll report back (about the conversion, not the cows).

- Bob