Module: Store Files on Disk

Re: Module: Store Files on Disk

cactux — Fri, 12 Jun 2020 21:04:41 -0400

It works now!
Not sure why, here's what I did:
1. Removed all modules which were not used, from the mods directory (all modules which were at Off).
2. Turned the module Off
3. Turned the module back at On.

Thanks for your help!

Re: Module: Store Files on Disk

Scott Finegan — Fri, 12 Jun 2020 09:00:27 -0400

If you turn the Store Files on Disk mod = OFF

Are the files uploaded correctly?

Re: Module: Store Files on Disk

cactux — Fri, 12 Jun 2020 02:43:20 -0400

Quote
Scott Finegan
Here is a whole thread on broken images [www.phorum.org]

Thanks. I'll try turning off modules to see if there's a conflict.

Re: Module: Store Files on Disk

cactux — Fri, 12 Jun 2020 02:40:58 -0400

Quote
Scott Finegan
Tidy is a Mod in the module directory.
There is also a php extension tidy, which may be disabled or enabled.

They are both Off.

Re: Module: Store Files on Disk

Scott Finegan — Thu, 11 Jun 2020 23:57:33 -0400

Quote

For your other suggestions:
> Tidy off.
In the Phorum admin I checked everything, did the sanity check, database integrity, everything.
Is that what you meant?

Tidy is a Mod in the module directory.
There is also a php extension tidy, which may be disabled or enabled.

Here is a whole thread on broken images [www.phorum.org]

I need to dig around for how to see the header info.

Re: Module: Store Files on Disk

cactux — Thu, 11 Jun 2020 16:24:36 -0400

Thanks Scott.

> Try a small file 20K, see if it works.
Here's what I tried: uploading 5 files of different sizes, from 10KB to 200KB.

local	server	ratio
200102	38217	5,236
153987	29172	5,279
100429	19005	5,284
50442	9684	5,209
20179	3853	5,237
10032	1834	5,470

Each time the size is smaller, and I calculated the ratio: always 5 something. Could this be a clue?

If I run the Linux command "file -i" on the files, I get:
1. For the original files: image/jpeg; charset=binary. And of course I can display them on my computer.
2. For the files stored on the server: application/octet-stream; charset=binary. And I can not open them with any software (Gimp, etc). It seems to be a binary garbage. The first chars are the same in each, but not the same than for a JPEG file.

For your other suggestions:
> Tidy off.
In the Phorum admin I checked everything, did the sanity check, database integrity, everything.
Is that what you meant?

> mime type has to be correct on the upload, check the header.
How do I do that? Where?

> Be sure time zone is set correctly. (PHP and server)
In php.ini this parameter was commented. I set it the same than the server. Still the same pb.

Thanks for your help, this is driving me crazy.

Re: Module: Store Files on Disk

Scott Finegan — Thu, 11 Jun 2020 10:12:20 -0400

Tidy off.
mime type has to be correct on the upload, check the header.
Be sure time zone is set correctly. (PHP and server)
Try a small file 20K, see if it works.

Re: Module: Store Files on Disk

cactux — Wed, 10 Jun 2020 23:01:48 -0400

Hi,

I am trying this module on my dev environment, and I have this problem: uploaded photos are smaller than original, and do not load. Max file size for attachments in Phorum settings is 250KB, much lower than in PHP settings (2M and 8M for post_max_size and upload_max_filesize).
I tried to upload various JPG files, with size between 90KB and 240KB, the files written on disk have a size of 34KB to 47KB and they do not load.

Any idea how to investigate? Thanks!

Re: Module: Store Files on Disk

cactux — Wed, 10 Jun 2020 22:58:05 -0400

Hi,

Quote
BryanB
How do we undo this module and revert back to having all the files stored in the database? Is there a script to force this back?

I just read the complete discussion looking for an answer to a question I have, and there's a message from Maurice that says no, this is not the goal of this module. It is a one way only.

Re: Module: Store Files on Disk

BryanB — Thu, 28 Nov 2019 08:49:42 -0500

How do we undo/revert this module back so the files are stored in the database rather than indexed and stored on the disk?

Re: Module: Store Files on Disk

BryanB — Thu, 28 Nov 2019 00:18:56 -0500

How do we undo this module and revert back to having all the files stored in the database? Is there a script to force this back?

Re: Converting files that are already stored in the database without console

Oliver Riesen-Mallmann — Thu, 10 Sep 2015 04:51:26 -0400

Hi Pavel,

I didn't look at the conversion script, but perhaps you can change it in a way that you can use it from your browser.

The main problem is, that the script surely needs too much execution time. So try to rewrite the script so that it is working in small portions.

Or check the script if it is possible to restart it over and over till all files a moved out of the database.

Regards
Oliver

Re: Converting files that are already stored in the database without console

Scott Finegan — Wed, 09 Sep 2015 20:28:07 -0400

Ask your host to run the console program.

Converting files that are already stored in the database without console

paw — Wed, 09 Sep 2015 11:58:18 -0400

Hi,
I install store_files_on_disk module and it works fine for new uploads. I have lot of files (massage attachments) stored in DB for years. I need to change a store mode because my provider setup limit for my DB size. I would like to convert all file from DB to dedicated directory within my web but I have not got console access to use the conversion script console_convert.php. I have got only ftp, PMA or standard possibilities from browser.
Is there some easy way how force movement files from DB to dedicated directory which is already used for store_files_on_disk?
Thanks for your help.
Pavel

Re: Module: Store Files on Disk

frost — Sat, 28 Jan 2012 17:43:48 -0500

Thank you very much.

Frost

Re: Module: Store Files on Disk

Thomas Seifert — Sat, 28 Jan 2012 17:40:44 -0500

Quote
Maurice Makaay

This module contains an on-the-fly conversion mechanism, which will convert files that are stored in the database to disk storage, right at the moment that they are requested. This mechanism will gradually convert the available files to the disk storage.

If you want to forcibly convert all available files to file storage, then you can use the conversion script console_convert.php that you can find in the module directory. This script's intended use it to run it from the system command prompt, using the PHP interpreter. See the module's README file for more info.

Re: Module: Store Files on Disk

frost — Sat, 28 Jan 2012 11:48:19 -0500

Hello,
a quick question: I am moving Phorum to a different server, but there is a limit on the size of the database. I have all the attachments stored in database, so that is 1.4GB of data. Is is possible to flush them out to disk using this plugin?
Thanks

Frost

Re: Module: Store Files on Disk

Maurice Makaay — Thu, 17 Feb 2011 02:58:52 -0500

Historically, we did not have a hook documentation system. This is something that we added a while ago. All new hooks are added to the code including the new documentation blocks. For old hooks, we need to write the documentation hook by hook. That has not yet been done for all hooks.

You are looking at "output from this module". That is not what you want to look at. You need to hook into the file retrieval process, not into a module that takes care of alternative storage. An on-the-fly watermarking module should be independent of the way in which the files are stored on the system.

The hook that you want to look into, is the "file_after_retrieve" hook. This one is called by the File API (well, the name kind of tells already ;-) after retrieving a file. You can find it in include/api/file.php at line 882. This is the hook that can be utilized for mangling file data before sending it to the client.

Re: Module: Store Files on Disk

starhawk — Wed, 16 Feb 2011 22:32:45 -0500

Hi Maurice,

No, I'm not to use this module to get watermarks, but the output from this module to feed to another that is supposed to generate watermarks.

Yes, I've tried that module a while ago but it's not working for me. Plus, even if it worked, it wouldn't have satisfied my needs where watermarks are generated and merged into the image files only when files are being retrieved, i.e. the files stored on the disk should be clean without watermarks, as opposed to marking the files once they're uploaded.

Wondering if there's any detailed manual exposing all known hooks and apis. The existing one on the web is somewhat incomplete.

Huge thanks.

Re: Module: Store Files on Disk

Maurice Makaay — Wed, 16 Feb 2011 00:22:20 -0500

Are you using this module to get the water marks done? That would be a bad idea. Water marking should be possible when using the default database-driven file storage too.

There is a water marking module. You might want to look at that one. I see some reports about it not working on some releases, but I don't know to what extends the issues were local for the people that reported them.

Re: Module: Store Files on Disk

starhawk — Tue, 15 Feb 2011 21:26:41 -0500

Just tried out the module which works pretty fine for me.

As I dig into the source code however. I found that the module utilizes hooks that are not documented in the developer manual. What I am looking for right now is a module that can consume the output from this one and add watermarks to image files and then output back to Phorum.

Maurice, could you please instruct if there's any existing modules to adopt or, if I have to implement it myself, any hooks I'd be using.

I'm new to Phorum. Please be as detailed as possible. Thanks.:)

Re: Module: Store Files on Disk

Maurice Makaay — Fri, 23 Apr 2010 16:44:44 -0400

Released version 1.1.1 of the module.

I changed the file path generation code to use gmstrftime() instead of strftime(). Because strftime() dependens on the server timezone, changing the server timezone resulted in changed file storage path and broken file downloads as result.

Files will be automatically converted to the new storage path on access. It is possible to use the console_convert.php script (which is a command line script) to speed up this process.

Note:
The automatic conversion from the 1.1.0 release will also be performed, so you can upgrade directly to this release when coming from a pre-1.1.0 version.

Do I have to upgrade?

You do not strictly have to upgrade, but it's advisable to do so, especially when you are going to move the forum to a different server (in which the timezone could differ from the timezone of the current server). When the server's timezone setting does not change, then there's no problem.

Re: Module: Store Files on Disk

Maurice Makaay — Fri, 23 Apr 2010 13:53:05 -0400

Only the root user on the system is allowed to change file ownership. However, the script does not have code in it to disallow access to the files when it's not owned by the correct user. As long as the files are readable for the web server, things should work.

Of course, it would not hurt if the files were owned by the web server user. But if you are hosting with a hosting provider, you'd have to request their support for help on making this work.

If you like, I could take a peek at your system. If you want me to do that, please PM me the ftp-server and account details. Shell access would be even better, but that's normally not provided with hosting accounts. Also provide an example URL where I can see a broken image in action.

Re: Module: Store Files on Disk

jero01 — Fri, 23 Apr 2010 13:37:53 -0400

I tried all the possible combinations of permissions but it still does not work.
I think the issue is ownership of the file.
Do you see anything in the script that would preclude display of the pic if it is not owned by 99?
Is it possible to change ownership of the pics to 99?

Re: Module: Store Files on Disk

Thomas Seifert — Fri, 23 Apr 2010 02:42:45 -0400

yes, that is an issue!
try to change the permission to rwx for "others" as thats what the system user is for you.

Re: Module: Store Files on Disk

jero01 — Fri, 23 Apr 2010 01:03:43 -0400

Thanks Maurice
I tried for an entire directory and subdirectory and all the files in there, but it still does not work.
The only thing I see is that the pictures, when sent through the module, are owned by 99 (the system?) while the pictures uploaded through FTP of my backup are owned by me, the user. Could that be an issue?

Re: Module: Store Files on Disk

Maurice Makaay — Thu, 22 Apr 2010 18:12:57 -0400

I don't think that the version of PHP and MySQL should be a factor here. Did you check the permissions on the subdirectories and files in those subdirectories?

Re: Module: Store Files on Disk

jero01 — Thu, 22 Apr 2010 15:28:40 -0400

Hello,

Thanks for your replies.
I checked on all your questions:
- Module is correctly enabled.
- Absolute path of the folder: nothing has changed
- Permissions on the folder are read-write
- I uploaded a new picture and verified that it went correctly to the directory (and not in the file_data field) and it is displaying correctly
Still all previous pictures are not displaying
The only point I just found out is that PHP and mysql were upgraded: could that be a factor?

Re: Module: Store Files on Disk

Maurice Makaay — Thu, 22 Apr 2010 03:49:33 -0400

I'm guessing the permissions part. Just uploading a new tree doesn't make things fully work. The directory, subdirectories and all files in it need to be readable and writable by the web server.

Another suspect would be the file storage path. If the server was exactly reconstructed or if the storage path was configured as a relative path, this should not be a problem. But if you used an absolute path, it could be that the new storage path differs from the original one. In that case, you'll have to update the module's settings.

You could check what happens when you upload a new attachment to a message. If the file contents end up in the files database, then it's clear that the webserver is unable to access the file storage directory. You can check the database to see if that happens: the file_data field will be filled in that case.

Re: Module: Store Files on Disk

Thomas Seifert — Thu, 22 Apr 2010 02:19:59 -0400

no permissions? no database entries? or you didn't enable the module.