All uploaded images broken after the upgrade

Quote
Scott Finegan
Hi Bill,

Try setting...

Language: PHP
error_reporting(-1);    or
error_reporting(E_ALL);

see [www.php.net]

Language: PHP
run   phpinfo();  on both installs and compare the output.  or
php -i on the command line and redirect to a file

... you are looking for missing GD, mbstring, Zip etc.

a


Ok I will check but I did compare the phpinfo and seem both similar only difference I can find is the php version itself that it. I will double check again.

Thank you

Bill

Quote
Scott Finegan
Hi Bill,

Try setting...

Language: PHP
error_reporting(-1);    or
error_reporting(E_ALL);

see [www.php.net]

Language: PHP
run   phpinfo();  on both installs and compare the output.  or
php -i on the command line and redirect to a file

... you are looking for missing GD, mbstring, Zip etc.

I checked both php and compared them, only thing I can see missing in 5.4 is "json" support. Do images in Phorum uses json functions?

Bill

json shouldn't be required.

You have probably already tried this, so fwiw.

Turn event logging module on.
Set Minimum log level to Debug
Software events ... check all of the boxes.


In the embed images module
What image viewer do you want to use for displaying full size images? Set to browser.

Advanced caching options.
Cache directory for scaled images (empty = use standard Phorum cache) Create a directory that is writable and readable like ./picturesx so you can easily find out if Phorum is doing something with it.
URL for scaled and cached images (empty = don't use feature) Leave empty for now. Then test again later with a path that should work.

Debugging problems
Platform support for the http_get API layer ... OK
Download test (tries to load [www.google.com]) Answer depends on allowed external links from forum or host.
Platform support for the image API layer ... OK (plus method used)
Enable debugging for this module ... Try it both ways while checking the broken images for error messages.

Check the Phorum event log.
Check the apache raw access log for result codes, and bad paths.
Check the php log.

Quote
Scott Finegan
json shouldn't be required.

You have probably already tried this, so fwiw.

Turn event logging module on.
Set Minimum log level to Debug
Software events ... check all of the boxes.


In the embed images module
What image viewer do you want to use for displaying full size images? Set to browser.

Advanced caching options.
Cache directory for scaled images (empty = use standard Phorum cache) Create a directory that is writable and readable like ./picturesx so you can easily find out if Phorum is doing something with it.
URL for scaled and cached images (empty = don't use feature) Leave empty for now. Then test again later with a path that should work.

Debugging problems
Platform support for the http_get API layer ... OK
Download test (tries to load [www.google.com]) Answer depends on allowed external links from forum or host.
Platform support for the image API layer ... OK (plus method used)
Enable debugging for this module ... Try it both ways while checking the broken images for error messages.

Check the Phorum event log.
Check the apache raw access log for result codes, and bad paths.
Check the php log.

I tried most of it, I don't think it the module itself because also avatar images isn't working either. I tried using it own cache folder and I know the module see the folder I created without problem because if I enter a folder name that don't exist the module produce an folder not found error when trying to save the setting.

For the error logs,

PHP log - none found related to images issues.

Apache log - none found related to images issue but strangely my banners having a relative path issue, it seem not working with relative path and I don't have anything in my .htaccess which could cause this issue. I even delete it and still having this relative path issue but I use absolute path then the banners would show up fine. Not sure if that related with images inside post messages.

Embedded Image debug reports
Image error
Debug info:
Load local file: 290884
Load local image file
Create thumbnail using 400x0 boundary
Creating thumbnail failed: Running getimagesize() on the image data failed

Event logging only reports that PHP notice: getimagesize(): Read error!

Raw error only complaining about the banner images outside of the post above the forums which couldn't detect images using relative path.

If I tried to access images directly inside the message, it only bring up a blank page in browser, no error message or symbol but it does produce a getimagesize() fail error in logs.

Again I don't think it the module or the relative path issue is effecting the Phorum because all messages and links are working fine just the images is not coming up at all or could not be found even it there in the folder.

I am using file on disk module by the way as I mentioned in my post earlier.

Any other suggestion for me to look into, I really want this resolved soon so I can move it all over to the new server before PHP 5.3 is at EOL.

Thank you for your assistance on this!

Bill

Here what I am doing right now ...

Creating a new Phorum forum on the same test server then I will try module one by one and see if it break anything in the process to narrow it down ...

I suspect the file on disk module but not sure yet since I cannot turn it off because all images had been converted to file on disk from database.

We have a huge forum where folks upload images everyday since the late 1990's so our collection is quite too huge to be stored in MYSQL database and when running backup it take forever to back it up that why we are using that file on disk module.

Bill

I created a new Phorum forum from scratch on same test server.

Installed the modules

embed_attachments
embed_images
store_files_on_disk
user_avatar

All images uploaded and working fine ...

I turn on Tidy and it broke the attachment "embedded" images but not the avatar images however on the other test site Tidy is turned off and still images are broken. I tried turning on and off Tidy but still did not restore the images.

I tried using PHP 5.5 instead and same result still.....

Now I am downgrading to PHP 5.3 on test server to see if that restore the images ....

Bill

Looks like there an issue with current images after the upgrade of PHP because if I start uploading images after the upgrade, the images are working fine

But the images before the upgrades still does not work just the ones uploaded after the PHP upgrade.

Anyone have any idea why is that?

Bill

What happens when you run the following script?

Try this on one of the pictures that is broken, and one that has been uploaded after the upgrade.
<?php
$file = '/home/yourweb/public_html/somedirectory/somepicture.jpg';
if (is_file($file)) {
$imginfo = getimagesize($file);
foreach ($imginfo as $key=>$val) {
echo $key . ' = ' . $val . '<br />';
}
} else {
echo 'The file was not found.';
}
?>

Quote
Scott Finegan
What happens when you run the following script?

Try this on one of the pictures that is broken, and one that has been uploaded after the upgrade.
<?php
$file = '/home/yourweb/public_html/somedirectory/somepicture.jpg';
if (is_file($file)) {
$imginfo = getimagesize($file);
foreach ($imginfo as $key=>$val) {
echo $key . ' = ' . $val . '<br />';
}
} else {
echo 'The file was not found.';
}
?>

Thanks for helping me troubleshoot on this ....

Here a link to the message post with the broken image I am testing it on. It the one on the far left.

[ns.findmall.com]


First test script code for that broken image

<?php 
$file = '/home/garyk/public_html/attachments/2014/0530/23/292038'; 
if (is_file($file)) { 
$imginfo = getimagesize($file); 
foreach ($imginfo as $key=>$val) { 
echo $key . ' = ' . $val . '<br />'; 
} 
} else { 
echo 'The file was not found.'; 
} 
?>

Here a link to the message post with the broken image I am testing it on. It the one on the far left.
[ns.findmall.com]

The link to the script [ns.findmall.com] and the output result

0 = 1024
1 = 768
2 = 2
3 = width="1024" height="768"
bits = 8
channels = 3
mime = image/jpeg







Second test script result on the working image

Link to the image which is working when uploaded and attached after the PHP upgrade.

[ns.findmall.com]

<?php 
$file = '/home/garyk/public_html/attachments/2014/0604/08/292042'; 
if (is_file($file)) { 
$imginfo = getimagesize($file); 
foreach ($imginfo as $key=>$val) { 
echo $key . ' = ' . $val . '<br />'; 
} 
} else { 
echo 'The file was not found.'; 
} 
?>

Link to the image script and the output for second test [ns.findmall.com]

0 = 204
1 = 204
2 = 2
3 = width="204" height="204"
bits = 8
channels = 3
mime = image/jpeg



Script seem to be able to see both images fine but the first image is still broken while second image which is uploaded after the PHP upgrade is working

What would be the next thing to troubleshoot?


Bill

If the broken image is the front and back of a coin, or two coins, the actual image is OK, I was able to navigate to it, download it, add .jpg and display it locally. A newly uploaded image works, so the Phorum code appears to be working.

Some things to investigate.

1) Are you using version: store_files_on_disk-1.1.1 ?
Are the timezone settings identical on all servers?

Quote

Changelog:
----------
2010-04-23 1.1.1

- Changed the file path generation code to use gmstrftime() instead
of strftime(). Because strftime() dependens on the server timezone,
changing the server timezone resulted in changed file storage path
and broken file downloads as result.

Quote

php 5.4 Backward Incompatible Changes
In the date and time extension, the timezone can no longer be set using the TZ environment variable. Instead you have to specify a timezone using the date.timezone php.ini option or date_default_timezone_set() function. PHP will no longer attempt to guess the timezone, and will instead fall back to "UTC" and issue a E_WARNING.

The phorum_messages table has a field named "meta" containing serialized data that indicates the file name, along with other data that may be unrelated to this issue as other mods use this field also.

The store_files_on_disk module adds a path "attachments" to the settings table, then gmstrftime() is used to generate subdirectories.


2) Please verify the tables are using the same character set on the test machine and your website?

I haven't checked lately, but a new database in MySQL creates the tables as CHARSET=latin1, unless you specify otherwise. Older Phorum installs may have tables created with DEFAULT CHARSET=utf8 or some other one.

When you load data into preexisting tables, whatever character set specified at table creation rules the day.


3) If it comes down to a php internal change, it may be simpler to create new minimal forums under the two versions of php, identical except php version, add one record each with an image, and compare the data that is stored.

Export the data from the old php/mysql, import to the latest php/mysql. Verify the image is broken. Add one new record with the same image, verify it is not broken. Compare the broken and not broken data and try to identify the the difference(s).

Quote
Scott Finegan
If the broken image is the front and back of a coin, or two coins, the actual image is OK, I was able to navigate to it, download it, add .jpg and display it locally. A newly uploaded image works, so the Phorum code appears to be working.

Some things to investigate.

1) Are you using version: store_files_on_disk-1.1.1 ?
Are the timezone settings identical on all servers?

Quote

Changelog:
----------
2010-04-23 1.1.1

- Changed the file path generation code to use gmstrftime() instead
of strftime(). Because strftime() dependens on the server timezone,
changing the server timezone resulted in changed file storage path
and broken file downloads as result.

Quote

php 5.4 Backward Incompatible Changes
In the date and time extension, the timezone can no longer be set using the TZ environment variable. Instead you have to specify a timezone using the date.timezone php.ini option or date_default_timezone_set() function. PHP will no longer attempt to guess the timezone, and will instead fall back to "UTC" and issue a E_WARNING.

The phorum_messages table has a field named "meta" containing serialized data that indicates the file name, along with other data that may be unrelated to this issue as other mods use this field also.

The store_files_on_disk module adds a path "attachments" to the settings table, then gmstrftime() is used to generate subdirectories.


2) Please verify the tables are using the same character set on the test machine and your website?

I haven't checked lately, but a new database in MySQL creates the tables as CHARSET=latin1, unless you specify otherwise. Older Phorum installs may have tables created with DEFAULT CHARSET=utf8 or some other one.

When you load data into preexisting tables, whatever character set specified at table creation rules the day.


3) If it comes down to a php internal change, it may be simpler to create new minimal forums under the two versions of php, identical except php version, add one record each with an image, and compare the data that is stored.

Export the data from the old php/mysql, import to the latest php/mysql. Verify the image is broken. Add one new record with the same image, verify it is not broken. Compare the broken and not broken data and try to identify the the difference(s).

1) I just upgraded the store on file to a newer version, still same result, both server are using the same timezone setting which is Central America "Chicago" timezone.

However I have not checked the time zone in php.ini yet .. I will check that one.

2) table on productive server and test server both using latin1


3) Data is exported from productive and imported into the test server which is the latest php/mysql , I did downgraded the php on the test server and it did not restore the image this time. So maybe it not the php but something else...

Bill

Quote
Scott Finegan
If the broken image is the front and back of a coin, or two coins, the actual image is OK, I was able to navigate to it, download it, add .jpg and display it locally. A newly uploaded image works, so the Phorum code appears to be working.

Some things to investigate.

1) Are you using version: store_files_on_disk-1.1.1 ?
Are the timezone settings identical on all servers?

Quote

Changelog:
----------
2010-04-23 1.1.1

- Changed the file path generation code to use gmstrftime() instead
of strftime(). Because strftime() dependens on the server timezone,
changing the server timezone resulted in changed file storage path
and broken file downloads as result.

Quote

php 5.4 Backward Incompatible Changes
In the date and time extension, the timezone can no longer be set using the TZ environment variable. Instead you have to specify a timezone using the date.timezone php.ini option or date_default_timezone_set() function. PHP will no longer attempt to guess the timezone, and will instead fall back to "UTC" and issue a E_WARNING.

The phorum_messages table has a field named "meta" containing serialized data that indicates the file name, along with other data that may be unrelated to this issue as other mods use this field also.

The store_files_on_disk module adds a path "attachments" to the settings table, then gmstrftime() is used to generate subdirectories.


2) Please verify the tables are using the same character set on the test machine and your website?

I haven't checked lately, but a new database in MySQL creates the tables as CHARSET=latin1, unless you specify otherwise. Older Phorum installs may have tables created with DEFAULT CHARSET=utf8 or some other one.

When you load data into preexisting tables, whatever character set specified at table creation rules the day.


3) If it comes down to a php internal change, it may be simpler to create new minimal forums under the two versions of php, identical except php version, add one record each with an image, and compare the data that is stored.

Export the data from the old php/mysql, import to the latest php/mysql. Verify the image is broken. Add one new record with the same image, verify it is not broken. Compare the broken and not broken data and try to identify the the difference(s).

Look like it's fixed!

Drumrolll ...............

You were right Scott ... It was the timezone issue but within the php.ini

On productive server in the php.ini the time zone was "date.timezone = "America/Chicago"" while the test server had it at "date.timezone = "UTC"" so when I changed it to "date.timezone = "America/Chicago"" all the images came back ... every one of them.


Thank you for all your feedback to troubleshoot on this.

Bill

Glad it is working.

I believe that the console_convert.php script will convert the paths to UTC now that it can locate, and move the files. You might run the script from a console and verify this. After this conversion, according to the changelog below, it should not matter which time zone is set.

Quote
Changelog
2010-04-23 1.1.1

- Changed the file path generation code to use gmstrftime() instead
of strftime(). Because strftime() dependens on the server timezone,
changing the server timezone resulted in changed file storage path
and broken file downloads as result.

Files will be automatically converted to the new storage path on
access. It is possible to use the console_convert.php script
(which is a command line script) to speed up this process.

Note:
The automatic conversion from the 1.1.0 release will also
be performed, so you can upgrade directly to this release when
coming from a pre-1.1.0 version.

howdy.

First off: this is Beautiful Coding. It's art, and I really mean it. The use of hooks: and templates is genius as a way to let pros and amateurs get what they want in looks and function. I am in awe, and having fun. I've written a module and working on a template and it's inspiring and enjoyable. thank you for this.

I also understand that there are just a few active followers here, so I'll be brief and to the point, and I'm posting here because it seems to be a very similar problem, though not with upgrading. This is a new forum, just built with the most recent download, and running off a hostgator server.

I'm seeing broken images in posts and nothing in emails (using Mailinglist or my own module). The error showing (with Embed Images and Embed Attachments and Error Logging on) shows

Image error
Debug info:
Load local file: 7
Load local image file
Create thumbnail using 500x500 boundary
Original image was 3715x1100
Thumbnail created at 500x148
Thumbnail MIME type: image/jpeg
OK 3715x1100 500x148
open image URL



and I can poke the open image URL and download the image just fine. I'm not seeing any php errors - though I might not be looking in the right place for this module's errors.

I suspect it might (hopefully) be me just missing an obvious setting (like setting the attachment size > 0, which it took me some time to find, ha!)....

Even if you don't have time for a comment, thanks again, not only for this open source project that so many use, but for allowing me to learn and have some fun at the same time.

many thanks

dale

PS if you'd like me to re-post as a new topic, please let me know - I posted here because it seems very symptomatically similar.

6/18/14 -- I'll try to attach an image here, just as I tried with my own phorum: (it worked here, so I'm suspecting a mis-setting?)

6/18/14: -- Turned off all other modules except logging and the two Embed_ mods. All checkboxes checked, have tried various viewers. Log shows no errors.
Noticed the Google upload was NOT OK... will look into your settings.php code for a clue next....

6/19/14: -- Next, I added a bit of code inside the Embed_Images settings.php to see if curl and sockets were available. They did show as existing, so I tried using curl to get the google page, and it worked (I simply echo'ed the results and contents which showed above the settings when it's run.)

So I'm still hunting for why phorum images aren't loading.

6/18/14 -- the "Not OK" message is due to a minor flaw in Embed_Image's settings.php code: it is checking for the returned google.com page by looking for the <html> tag, and google doesn't always use it that way. for instance, today it has <html itemscope="" itemtype="[schema.org]; lang="en">. You could change the check by loosing the closing ">", I think. Still can't figure out why I'm seeing all broken images.

.



Edited 5 time(s). Last edit at 06/19/2014 08:55PM by dmcbeath.

What are you using for a picture cache directory?

From my admin ... embed_module ...

Advanced caching options. See help info for security notices.
Cache directory for scaled images (empty = use standard Phorum cache) ...  ./picturesx
URL for scaled and cached images (empty = don't use feature) ... http : //www.xxxxxxxxx.c0m/phorum/picturesx/

The first option keeps the files separate from the other cache stuff.

The second second gives me a click-able link directly to the thumbnail. This click-able link is where Phorum put the file (or tried to).
If you check the picture cache directory (ftp), or the Hostgator panel you should find the file at that location. If the file exists, and the link does not work, check the file and directory permissions. 644 should work (but then this is Hostgator). Check the apache raw access log for that file name, and the error code. Ask Hostgator

If the file does not exist.
Check the php error log.

Quote

Noticed the Google upload was NOT OK... will look into your settings.php code for a clue next....

There are many reasons for this not to work. 1) hostgator security. 2) phorum is not allowing links to external "files". 3)google may block

For the email not working check with hostgator, many hosts have locked down email by default, to limit spam.

Thanks for the response. I did find the solution just this morning.

1) the Embed_Images settings Download Test tries to load google.com for a returned page containing a text string "<html>", and unfortunately, google.com doesn't always use that tag. for instance, today it looks like this:

 
<html itemscope="" itemtype="[schema.org]; lang="en">

2) if the Embed_Images setting Enable Debugging for the module is YES, then I always see the broken image box with debug info, even if I fix what's causing the problem and everything is working fine. So I was just missing the point that it was actually working fine.

I have to say, it's been a real pleasure digging through the code. A great learning experience, both in elegance and technique.


Thanks!

dale



Edited 3 time(s). Last edit at 06/22/2014 10:58PM by dmcbeath.

Quote
olii
In file.php you can add a line

while(ob_get_level()) ob_end_clean();

before

$flags = empty($PHORUM['args']['download'])
? 0 : PHORUM_FLAG_FORCE_DOWNLOAD;
phorum_api_file_send($file, $flags);

This fixed it for me running > 5.5

Going to need more to go on than "it doesn't work".

Hi,

i still think it's only a problem in function phorum_ob_clean in common.php.

I proposed a patch: [github.com]

Regards
Oliver

---

Using Phorum since 7/2000: forum.langzeittest.de (actual version 5.2.23)
Modules "Made in Germany" for version 5.2: Author_as_Sender, CarCost, Close_Topic, Conceal_Message_Timestamp,
Format_Email, Index_Structure, Mailing_List, Pervasive_Forum, Spritmonitor, Terms_of_Service and German_Language_Files_Package.

Oliver,

(This probably applies to php 5.5.x as that was the version referenced in this thread)

Under php 5.6.0 your proposed patch fixed my undisplayable images.
The thumbnails were made correctly, regardless of the presence of the patch.

Photos uploaded after 5.6.0 upgrade but before patch were corrupt, and needed to be re uploaded. Photos uploaded before 5.6.0 displayed ok after patch.

Hello,

I have a similar problem but I don't understand how image can be corrupted ? All images are lost ? :'(

Hi nnx,

which version of Phorum you are using?

Regards
Oliver

---

Using Phorum since 7/2000: forum.langzeittest.de (actual version 5.2.23)
Modules "Made in Germany" for version 5.2: Author_as_Sender, CarCost, Close_Topic, Conceal_Message_Timestamp,
Format_Email, Index_Structure, Mailing_List, Pervasive_Forum, Spritmonitor, Terms_of_Service and German_Language_Files_Package.

Hello,

Few weeks ago I upgrade phorum 5.2.20 to 5.2.21, and to do that I thought I have to upgrade php from 4.4 to 5.x. I choose php 5.6 (the last supported version before 7).

I also use "store_files_on_disk" mod, the last from many years.

I thought Phorum were ok but it's not. All images are broken.

During 2 nights I did many test to debug Phorum scripts and find the root cause of my problem.

I can't go back to php4 with Phorum 5.2.21. I try many test to understand what's going on. I add many "echo" in api/files.php and there are my results.

With PHP 5.6 I saw that (see attachments). Filepath is correct, myme-type is ok, size seems to be ok, and the content is file-data string.

I go back to PHP 5.4, because I'm just curious and there is the result (see attachments).

There is no more special caracter...

I thought it's was the solution but no... I delete all "echo" and the problem is the same : images are broken.

It's like the stream is corrupted, but I don't understand why... I don't find php function that return an error.

I saw your message about a patch with "common.php" to use with PHP >= 5.6. So I undo the modification, just to test, but nothing change.

With php 5.6 and the last common.php version : I upload image and it works.
With php 5.4 and the old or the last common.php : I upload images, it's works, and image uploaded with php 5.6 is still working.

Now I go back to php 5.6, with all original phorum files, and I don't know why images are broken. Maybe something different between php 4.4 and php 5.X but what ??

Hi,

Any idea ? :'(

If you link directly to any of the files in the storage directory, they display correctly in your browser ?


Grasping at straws... read the manual.
Looking at your debug images above: Many ??? frequently indicate character set problems.
I did not do a thorough search for character set changes between php 4.4 and 5.6, however...

From http://php.net/manual/en/migration56.incompatible.php

Quote

User Contributed Notes
Someone
Beware, now default_charset in PHP 5.6 is UTF-8, instead of blank value like in previous versions. So if you use 8-bit charset like iso-8859-1 then your non-ascii characters will be broken until you explicitly set default_charset to blank value or to your actual character set. HTML meta tag doesn't work anymore because HTTP headers have more proirity over them. And Apache AddDefaultCharset directive has lower proirity too.

Edited 1 time(s). Last edit at 10/30/2016 07:18PM by Scott Finegan.

Hello,

I did this test with 2 files.

The first file : /www/forums/posted_files/2011/1217/09/309
It's an old image broken since php (or phorum 5.2.21) update

The second file : /www/forums/posted_files/2016/1023/16/601
A new file, uploaded recently and it's working in phorum.

I tested url in my browser. In all cases, Firefox offer me to download file and show this type "application/octet-stream".

Under Windows, I tried to open 601 file with photo viewer : it works.

I tried to open 309 file and it failed. Image seems to be corrupted.

I try to add file extension : jpg for the first file, and png for the second, and results are the same.

It's like php doesn't write file using the same technique between php 4.4 and php 5.4...

Do you know if default-charset change can explain this ? I'm lost...

Edit : After some control, I update php from 5.2 to 5.6, and not 4.4 to 5.6.



Edited 2 time(s). Last edit at 11/01/2016 05:12AM by nnx.

I try to put this line :

Language: PHP
ini_set( ';default_charset';, ';ISO-8859-1'; );

just before "file_get_contents" in store_files_on_disk.php.

It may be a problem of character set

This test failed...

Thanks for you help Scott.

I don't believe it is a problem with character set, unless at the server or internet server level.
Your direct link to 309 retrieves a corrupt file.

You need to recreate the old server, old phorum environment and test.
What version of "store_files_on_disk" mod did you use?

Do you have a backup of the photos?
Check some of those on a local computer to see if they display.

Hi,

I use last version of Phorum and mods.

I have several backup, and it's the same result under windows : I can't view my image in Windows...

I don't understand how a php upgrade can corrupt all files...

It's like files were encoded as string with a charset, and read with another...