Firefox PHP

Problems + fixes upgrading from Phorum 3 to 5.2.8

Posted by Simon King 
Problems + fixes upgrading from Phorum 3 to 5.2.8
August 03, 2008 01:14AM
I am the administrator for some very old forums running Phorum 3.2.11 which I am now in the process of upgrading to Phorum 5.2.8. This series of active forums has well over 1 million posts so I'm taking my time on the upgrade, testing it piece by piece until I'm ready to flip the switch and do it for real. As you already know it requires me to first upgrade to 3.4.8a and then to 5.2.8 and I'm sure I'll learn many things along the way that I hope to share here if they seem valuable to others. I'll also probably be stopping by for help when I run into sticky situations converting the custom hacks of my old forum into mods for the the new one.

Yesterday I ran my first test upgrade on one of the smaller forums (only 2500 mgs) and ran into a couple of problems:

First, my attachments were not being added to the phorum_files table. When viewing the forum after the upgrade it knew which messages had attachments, but clicking on them generated an error message (since they weren't actually added to the db). I tracked the problem down and fixed it by replacing line 298 in the /scripts/phorum3_in.php file:

old code
Language: PHP
  1. $file_id = phorum_db_file_save($userid, $attachment['filename'], filesize($filename), $buffer, newmessage['message_id']);

new code
Language: PHP
  1. $file = array("user_id" => $userid, "filename" => $attachment['filename'], "filesize" => filesize($filename), "file_data" => $buffer, "message_id" => $newmessage['message_id'], "link" => PHORUM_LINK_MESSAGE);
  2. $file_id = phorum_db_file_save($file);

Also, at one point the upgrade scripts started spitting out junk characters (looked like base64 encoded binary) to the screen and then ended with an uninformative error. It turns out that this was caused by an attachment that was larger than the upper limit allowed by my database server. After increasing the max_allowed_packet value in the my.cnf file on my server it solved this issue, thanks to Maurice for posting about that earlier. If the script was able to detect the max_allowed_packet value and provide an appropriate error message and clean exit when encountering a file that is too large it would have saved me some time. Perhaps a simpler fix is to include information about this setting in the /scripts/phorum3to5convert.php file and online documentation.

Despite these issues I'm happy with the upgrade process so far and want to say thank you for supporting an upgrade path for users like me to have been running old Phorum boards for a long time. I'll keep you posted as I slowly continue to test this upgrade.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
August 04, 2008 07:59PM
Thanks a lot for the patch. Apparently, we forgot to update the upgrade script for the new 5.2 database layer. I added the update to our code in changeset 3478.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
August 14, 2008 10:17PM
Well, I sure came to the right place. I'm in almost exactly the same boat. I'm just now starting an upgrade of a very large 3.2.11 Phorum. I don't have a million posts, but there are a couple hundred thousand. I'll also add my experiences here. Now, off to upgrade to 3.4.8a.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
August 15, 2008 03:39AM
Well, I'm having a problem at the very first step--installing 3.4.8a. I've uploaded all the files to the server, but when I run /forum/admin/index.php, all I get is a blank screen. No error messages, just a blank screen. I'm hoping this is something simple. I searched through the Phorum 3 board, but didn't find any solution to this problem.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
August 15, 2008 07:35AM
The place to look for error messages would be the server's error logging. This is something that crashes PHP, e.g. a typo in the configuration. Hard to tell what's going on without the exact error.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
August 15, 2008 01:15PM
The only error message I get is:

[Fri Aug 15 06:58:57 2008] [error] [client 75.46.19.127] File does not exist: /home/flxweb/public_html/404.shtml, referer: <domain>/forum/admin/index.php

where <domain> is my domain name. When I substitute a simple HTML file for index.php, it runs just fine, so maybe one of the includes is not finding a file it needs to include?
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
August 15, 2008 01:34PM
well, like maurice said, it could be a simple parse error in a config-file or the like. testing a html file which doesn't call any parsers isn't helping at all.
the error message you posted about is from the webserver error log, try to find a php error log or something like that.
a blank page isn't telling much about the error unfortunately. I can only recommend to go over the installation instructions again.


Thomas Seifert
Phorum Development Team / Mysnip-Solutions.de
Custom Phorum and general software development
worry-free Phorum Hosting
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
August 15, 2008 03:06PM
I also had some show stopping problems upgrading to the latest 3.x version. It's hard to say if it's the same issue but I fixed mine by changing that directory to run through PHP4 instead of PHP5. If you have both installed this can be done with a simple .htaccess command, just Google for the syntax.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
October 11, 2008 09:36PM
I am still working on my phorum upgrade. My main problem is that the script never makes it through all my forums, halting with different problems and at different points. I'm quite sure it has to do with system load or memory issues, even though I have included ini_set statements that increase the allowed memory. I'm proceeding with trial and error.

I have over 14GB of attachments to upgrade, and I think that's the crux of the problem. I am running an upgrade test right now to see if putting the attachments on the file system rather than the database will fix my issues. I prefer to have them on the disk anyway, so even if this isn't an issue it's a step in the right direction for me.

I wanted to point out another change in the /scripts/phorum3_in.php file that I had to make in order to use the Store Files on Disk module. Basically I just needed to change the db storage call to use the new api file storage instead.

Language: PHP
  1. $file_id = phorum_api_file_store($file);

Of course this also requires the /include/api/file_storage.php file to be included, which I chose to do within /phorum3to5convert.php

Language: PHP
require_once("./include/api/file_storage.php");

From my perspective this seems like a change that should be made to the upgrade files in the main distribution. Hopefully this will make things easier in the future for people who want to install the Store Files on Disk module before upgrading.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
October 12, 2008 12:23AM
Thanks, I think that makes sense indeed. The file storage API was created after the conversion script, so it never got into that one. Additionally, we never figured that people would start out with file storage :)

I updated the scripts in changeset 3555.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
October 12, 2008 08:21AM
Thanks for the quick update, but as I played with this more tonight I realized that converting the script to work with the API requires a couple more changes to /scripts/phorum3_in.php.

1. The file data should not be base64 encoded before passing it along to phorum_api_file_store. This will be done later if saving to the db but it causes the attachment to be stored on the file system in the encoded format, which causes problems later when retrieving the file. So, the following line needed to change:

Language: PHP
$buffer=base64_encode(fread($fp, filesize($filename)));

to

Language: PHP
$buffer=fread($fp, filesize($filename));

2. The API call returns a file array, not a numerical file id. Because of this the attachment meta data that was being saved in the database was getting messed up. This can be fixed by changing the following line of code:

Language: PHP
$inserted_files[]=array("file_id"=>$file_id, "name"=>$attachment['filename'], "size"=>filesize($filename));

to

Language: PHP
$inserted_files[]=array("file_id"=>$file_id['file_id'], "name"=>$attachment['filename'], "size"=>filesize($filename));

The naming of $file_id is confusing now because it is actually retuning an array of which the id is only one element. To avoid confusion I think the variable name should be changed to something like $stored_file instead.

After making these changes I upgraded one of my smaller forums and confirmed that attachments were properly saved to the file system and load correctly when accessed via the forum. For the sake of clarity here is the complete code block that I changed to make this script use the API:

Language: PHP
$fp=fopen($filename, "r"); $buffer=fread($fp, filesize($filename)); fclose($fp); $file = array("user_id" => $userid, "filename" => $attachment['filename'], "filesize" => filesize($filename), "file_data" => $buffer, "message_id" => $newmessage['message_id'], "link" => PHORUM_LINK_MESSAGE); $stored_file = phorum_api_file_store($file); unset($buffer); // free that large buffer $inserted_files[]=array("file_id"=>$stored_file['file_id'], "name"=>$attachment['filename'], "size"=>filesize($filename));
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8
October 12, 2008 12:09PM
Thanks. I put the additional updates in the code too.
See changeset 3556.


Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Sorry, only registered users may post in this forum.

Click here to login