Problems + fixes upgrading from Phorum 3 to 5.2.8
Posted by Simon King
Problems + fixes upgrading from Phorum 3 to 5.2.8 August 02, 2008 07:14PM |
Registered: 19 years ago Posts: 24 |
I am the administrator for some very old forums running Phorum 3.2.11 which I am now in the process of upgrading to Phorum 5.2.8. This series of active forums has well over 1 million posts so I'm taking my time on the upgrade, testing it piece by piece until I'm ready to flip the switch and do it for real. As you already know it requires me to first upgrade to 3.4.8a and then to 5.2.8 and I'm sure I'll learn many things along the way that I hope to share here if they seem valuable to others. I'll also probably be stopping by for help when I run into sticky situations converting the custom hacks of my old forum into mods for the the new one.
Yesterday I ran my first test upgrade on one of the smaller forums (only 2500 mgs) and ran into a couple of problems:
First, my attachments were not being added to the phorum_files table. When viewing the forum after the upgrade it knew which messages had attachments, but clicking on them generated an error message (since they weren't actually added to the db). I tracked the problem down and fixed it by replacing line 298 in the /scripts/phorum3_in.php file:
old code
new code
Also, at one point the upgrade scripts started spitting out junk characters (looked like base64 encoded binary) to the screen and then ended with an uninformative error. It turns out that this was caused by an attachment that was larger than the upper limit allowed by my database server. After increasing the max_allowed_packet value in the my.cnf file on my server it solved this issue, thanks to Maurice for posting about that earlier. If the script was able to detect the max_allowed_packet value and provide an appropriate error message and clean exit when encountering a file that is too large it would have saved me some time. Perhaps a simpler fix is to include information about this setting in the /scripts/phorum3to5convert.php file and online documentation.
Despite these issues I'm happy with the upgrade process so far and want to say thank you for supporting an upgrade path for users like me to have been running old Phorum boards for a long time. I'll keep you posted as I slowly continue to test this upgrade.
Yesterday I ran my first test upgrade on one of the smaller forums (only 2500 mgs) and ran into a couple of problems:
First, my attachments were not being added to the phorum_files table. When viewing the forum after the upgrade it knew which messages had attachments, but clicking on them generated an error message (since they weren't actually added to the db). I tracked the problem down and fixed it by replacing line 298 in the /scripts/phorum3_in.php file:
old code
Language: PHP
$file_id = phorum_db_file_save($userid, $attachment[';filename';], filesize($filename), $buffer, newmessage[';message_id';]);
new code
Language: PHP
$file_id = phorum_db_file_save($file);
Also, at one point the upgrade scripts started spitting out junk characters (looked like base64 encoded binary) to the screen and then ended with an uninformative error. It turns out that this was caused by an attachment that was larger than the upper limit allowed by my database server. After increasing the max_allowed_packet value in the my.cnf file on my server it solved this issue, thanks to Maurice for posting about that earlier. If the script was able to detect the max_allowed_packet value and provide an appropriate error message and clean exit when encountering a file that is too large it would have saved me some time. Perhaps a simpler fix is to include information about this setting in the /scripts/phorum3to5convert.php file and online documentation.
Despite these issues I'm happy with the upgrade process so far and want to say thank you for supporting an upgrade path for users like me to have been running old Phorum boards for a long time. I'll keep you posted as I slowly continue to test this upgrade.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 August 04, 2008 01:59PM |
Admin Registered: 19 years ago Posts: 8,532 |
Thanks a lot for the patch. Apparently, we forgot to update the upgrade script for the new 5.2 database layer. I added the update to our code in changeset 3478.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 August 14, 2008 04:17PM |
Registered: 18 years ago Posts: 21 |
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 August 14, 2008 09:39PM |
Registered: 18 years ago Posts: 21 |
Well, I'm having a problem at the very first step--installing 3.4.8a. I've uploaded all the files to the server, but when I run /forum/admin/index.php, all I get is a blank screen. No error messages, just a blank screen. I'm hoping this is something simple. I searched through the Phorum 3 board, but didn't find any solution to this problem.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 August 15, 2008 01:35AM |
Admin Registered: 19 years ago Posts: 8,532 |
The place to look for error messages would be the server's error logging. This is something that crashes PHP, e.g. a typo in the configuration. Hard to tell what's going on without the exact error.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 August 15, 2008 07:15AM |
Registered: 18 years ago Posts: 21 |
The only error message I get is:
[Fri Aug 15 06:58:57 2008] [error] [client 75.46.19.127] File does not exist: /home/flxweb/public_html/404.shtml, referer: <domain>/forum/admin/index.php
where <domain> is my domain name. When I substitute a simple HTML file for index.php, it runs just fine, so maybe one of the includes is not finding a file it needs to include?
[Fri Aug 15 06:58:57 2008] [error] [client 75.46.19.127] File does not exist: /home/flxweb/public_html/404.shtml, referer: <domain>/forum/admin/index.php
where <domain> is my domain name. When I substitute a simple HTML file for index.php, it runs just fine, so maybe one of the includes is not finding a file it needs to include?
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 August 15, 2008 07:34AM |
Admin Registered: 21 years ago Posts: 9,240 |
well, like maurice said, it could be a simple parse error in a config-file or the like. testing a html file which doesn't call any parsers isn't helping at all.
the error message you posted about is from the webserver error log, try to find a php error log or something like that.
a blank page isn't telling much about the error unfortunately. I can only recommend to go over the installation instructions again.
Thomas Seifert
the error message you posted about is from the webserver error log, try to find a php error log or something like that.
a blank page isn't telling much about the error unfortunately. I can only recommend to go over the installation instructions again.
Thomas Seifert
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 August 15, 2008 09:06AM |
Registered: 19 years ago Posts: 24 |
I also had some show stopping problems upgrading to the latest 3.x version. It's hard to say if it's the same issue but I fixed mine by changing that directory to run through PHP4 instead of PHP5. If you have both installed this can be done with a simple .htaccess command, just Google for the syntax.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 October 11, 2008 03:36PM |
Registered: 19 years ago Posts: 24 |
I am still working on my phorum upgrade. My main problem is that the script never makes it through all my forums, halting with different problems and at different points. I'm quite sure it has to do with system load or memory issues, even though I have included ini_set statements that increase the allowed memory. I'm proceeding with trial and error.
I have over 14GB of attachments to upgrade, and I think that's the crux of the problem. I am running an upgrade test right now to see if putting the attachments on the file system rather than the database will fix my issues. I prefer to have them on the disk anyway, so even if this isn't an issue it's a step in the right direction for me.
I wanted to point out another change in the /scripts/phorum3_in.php file that I had to make in order to use the Store Files on Disk module. Basically I just needed to change the db storage call to use the new api file storage instead.
Of course this also requires the /include/api/file_storage.php file to be included, which I chose to do within /phorum3to5convert.php
From my perspective this seems like a change that should be made to the upgrade files in the main distribution. Hopefully this will make things easier in the future for people who want to install the Store Files on Disk module before upgrading.
I have over 14GB of attachments to upgrade, and I think that's the crux of the problem. I am running an upgrade test right now to see if putting the attachments on the file system rather than the database will fix my issues. I prefer to have them on the disk anyway, so even if this isn't an issue it's a step in the right direction for me.
I wanted to point out another change in the /scripts/phorum3_in.php file that I had to make in order to use the Store Files on Disk module. Basically I just needed to change the db storage call to use the new api file storage instead.
Language: PHP
$file_id = phorum_api_file_store($file);
Of course this also requires the /include/api/file_storage.php file to be included, which I chose to do within /phorum3to5convert.php
Language: PHPrequire_once("./include/api/file_storage.php");
From my perspective this seems like a change that should be made to the upgrade files in the main distribution. Hopefully this will make things easier in the future for people who want to install the Store Files on Disk module before upgrading.
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 October 11, 2008 06:23PM |
Admin Registered: 19 years ago Posts: 8,532 |
Thanks, I think that makes sense indeed. The file storage API was created after the conversion script, so it never got into that one. Additionally, we never figured that people would start out with file storage :)
I updated the scripts in changeset 3555.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
I updated the scripts in changeset 3555.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 October 12, 2008 02:21AM |
Registered: 19 years ago Posts: 24 |
Thanks for the quick update, but as I played with this more tonight I realized that converting the script to work with the API requires a couple more changes to /scripts/phorum3_in.php.
1. The file data should not be base64 encoded before passing it along to phorum_api_file_store. This will be done later if saving to the db but it causes the attachment to be stored on the file system in the encoded format, which causes problems later when retrieving the file. So, the following line needed to change:
to
2. The API call returns a file array, not a numerical file id. Because of this the attachment meta data that was being saved in the database was getting messed up. This can be fixed by changing the following line of code:
to
The naming of $file_id is confusing now because it is actually retuning an array of which the id is only one element. To avoid confusion I think the variable name should be changed to something like $stored_file instead.
After making these changes I upgraded one of my smaller forums and confirmed that attachments were properly saved to the file system and load correctly when accessed via the forum. For the sake of clarity here is the complete code block that I changed to make this script use the API:
1. The file data should not be base64 encoded before passing it along to phorum_api_file_store. This will be done later if saving to the db but it causes the attachment to be stored on the file system in the encoded format, which causes problems later when retrieving the file. So, the following line needed to change:
Language: PHP$buffer=base64_encode(fread($fp, filesize($filename)));
to
Language: PHP$buffer=fread($fp, filesize($filename));
2. The API call returns a file array, not a numerical file id. Because of this the attachment meta data that was being saved in the database was getting messed up. This can be fixed by changing the following line of code:
Language: PHP$inserted_files[]=array("file_id"=>$file_id, "name"=>$attachment[';filename';], "size"=>filesize($filename));
to
Language: PHP$inserted_files[]=array("file_id"=>$file_id[';file_id';], "name"=>$attachment[';filename';], "size"=>filesize($filename));
The naming of $file_id is confusing now because it is actually retuning an array of which the id is only one element. To avoid confusion I think the variable name should be changed to something like $stored_file instead.
After making these changes I upgraded one of my smaller forums and confirmed that attachments were properly saved to the file system and load correctly when accessed via the forum. For the sake of clarity here is the complete code block that I changed to make this script use the API:
Language: PHP$fp=fopen($filename, "r"); $buffer=fread($fp, filesize($filename)); fclose($fp); $file = array("user_id" => $userid, "filename" => $attachment[';filename';], "filesize" => filesize($filename), "file_data" => $buffer, "message_id" => $newmessage[';message_id';], "link" => PHORUM_LINK_MESSAGE); $stored_file = phorum_api_file_store($file); unset($buffer); // free that large buffer $inserted_files[]=array("file_id"=>$stored_file[';file_id';], "name"=>$attachment[';filename';], "size"=>filesize($filename));
Re: Problems + fixes upgrading from Phorum 3 to 5.2.8 October 12, 2008 06:09AM |
Admin Registered: 19 years ago Posts: 8,532 |
Thanks. I put the additional updates in the code too.
See changeset 3556.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
See changeset 3556.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Sorry, only registered users may post in this forum.