make phorum spider friendly
Posted by Christian Springub
make phorum spider friendly August 30, 2001 03:47PM |
Hi,
this hack will make the articels in your forum spider friendly. The script doesn't change the phorum code, it generates a seperate spider friendly index of all messages.
( For people how doesn't know, why spiders don't take phorum sites in their index should read this article: [www.zend.com])
To use this script you need the mod_rewrite for apache (http://httpd.apache.org/docs/mod/mod_rewrite.html)
What this script does:
The script generate index of all your phorum messages. With the mod_rewrite it will transform the name of the files
direct on the server, without possibility for the spider to notice that. An example: The index will generate a link like
this one: topic-5-342-342.html. This is fine for a spider. The mod_rewrite will rewrite it to read.php?f=5&i=342&t=342.
That's it.
How to install it:
1) First you have to check, if you have the mod_rewrite installed. If not, ask your hoster.
2) Add the following lines to your .htaccess file. (if you haven't anyone, create a new one):
-- insert this into .htaccess
# Start the rewrite engine
RewriteEngine on
# Set the rule for rewriting like in the example
RewriteRule ^topic-(.*)-(.*)-(.*)\.html$ read.php?f=$1&i=$2&t=$3
# Set the rule for rewrite the files of the index (for example: topics-20.html -> topics.php$start=20)
RewriteRule ^topics-(.*)\.html$ topics.php?start=$1
--
3) Edit the configuration section in the script
4) create a file called topics.html (attention, not .php) and put the script into it.
4) Upload it. Open topics-0.html. Enjoy it.
5) I just write this hack and would be glad about any form of feedback. Please mail me, if you want to use this script. If enough people are interested, I will not hesitate to improve the script.
mailto: springub@dream-up.de
A working example:
[www.manic-depressive.de] (pay attention at the links)
Greetings,
Christian Springub
this hack will make the articels in your forum spider friendly. The script doesn't change the phorum code, it generates a seperate spider friendly index of all messages.
( For people how doesn't know, why spiders don't take phorum sites in their index should read this article: [www.zend.com])
To use this script you need the mod_rewrite for apache (http://httpd.apache.org/docs/mod/mod_rewrite.html)
What this script does:
The script generate index of all your phorum messages. With the mod_rewrite it will transform the name of the files
direct on the server, without possibility for the spider to notice that. An example: The index will generate a link like
this one: topic-5-342-342.html. This is fine for a spider. The mod_rewrite will rewrite it to read.php?f=5&i=342&t=342.
That's it.
How to install it:
1) First you have to check, if you have the mod_rewrite installed. If not, ask your hoster.
2) Add the following lines to your .htaccess file. (if you haven't anyone, create a new one):
-- insert this into .htaccess
# Start the rewrite engine
RewriteEngine on
# Set the rule for rewriting like in the example
RewriteRule ^topic-(.*)-(.*)-(.*)\.html$ read.php?f=$1&i=$2&t=$3
# Set the rule for rewrite the files of the index (for example: topics-20.html -> topics.php$start=20)
RewriteRule ^topics-(.*)\.html$ topics.php?start=$1
--
3) Edit the configuration section in the script
4) create a file called topics.html (attention, not .php) and put the script into it.
4) Upload it. Open topics-0.html. Enjoy it.
5) I just write this hack and would be glad about any form of feedback. Please mail me, if you want to use this script. If enough people are interested, I will not hesitate to improve the script.
mailto: springub@dream-up.de
A working example:
[www.manic-depressive.de] (pay attention at the links)
Greetings,
Christian Springub
Re: make phorum spider friendly August 30, 2001 03:53PM |
Re: make phorum spider friendly September 25, 2001 06:44AM |
Re: make phorum spider friendly September 25, 2001 09:30AM |
Hi,
yes, there is an hack in action:
[www.manic-depressive.de]
(pay attention to the links, there are only .html links without parameters like ?f=5)
Greetings,
Christian Springub
yes, there is an hack in action:
[www.manic-depressive.de]
(pay attention to the links, there are only .html links without parameters like ?f=5)
Greetings,
Christian Springub
Re: make phorum spider friendly September 29, 2001 07:08PM |
Hey,
I think, I'll use this on jugendserver.spinnenwerk.de but i had some troubles getting it running:
so i changed your line:
RewriteRule ^topic-(.*)-(.*)-(.*)\.html$ read.php?f=$1&i=2&t=$3
to:
RewriteRule ^topic-(.*)-(.*)-(.*)\.html$ read.php?f=$1\&i=2\&t=$3
the slashes do a good job now ;)
Otherwise apache takes them as regex I think...
Bye.
I think, I'll use this on jugendserver.spinnenwerk.de but i had some troubles getting it running:
so i changed your line:
RewriteRule ^topic-(.*)-(.*)-(.*)\.html$ read.php?f=$1&i=2&t=$3
to:
RewriteRule ^topic-(.*)-(.*)-(.*)\.html$ read.php?f=$1\&i=2\&t=$3
the slashes do a good job now ;)
Otherwise apache takes them as regex I think...
Bye.
Re: make phorum spider friendly May 08, 2002 11:40PM |
Registered: 22 years ago Posts: 16 |
Re: make phorum spider friendly May 09, 2002 06:08AM |
Re: make phorum spider friendly May 11, 2002 04:12PM |
Registered: 22 years ago Posts: 16 |
Christian, there are a few items not covered in your instructions.
1) Into which .htaccess file do we insert the Rewrite statements? There are .htaccess files scattered all over the place but none in the Phorum top level.
2) Within topics.txt, you forget to include the step which discusses creating topics.html. The script itself is saved *AS* topics.html?
3) To which directory does topics.html get uploaded?
1) Into which .htaccess file do we insert the Rewrite statements? There are .htaccess files scattered all over the place but none in the Phorum top level.
2) Within topics.txt, you forget to include the step which discusses creating topics.html. The script itself is saved *AS* topics.html?
3) To which directory does topics.html get uploaded?
Re: make phorum spider friendly May 12, 2002 01:22PM |
Hi,
> 1) Into which .htaccess file do we insert the Rewrite
> statements? There are .htaccess files scattered all over the
> place but none in the Phorum top level.
Just create a new one in the directory you place the script.
> 2) Within topics.txt, you forget to include the step which
> discusses creating topics.html. The script itself is saved
> *AS* topics.html?
There ist nothing special. The topics.txt just generates the list with the links. You can take any html file and place the topics.txt (place <? at the beginning and ?> at the end :) ) in it. See the example at:
[www.manic-depressive.de]
The list is generated by the script.
> 3) To which directory does topics.html get uploaded?
that is your decision. Place it whereever you want.
Do you have any problems by using the script?
> 1) Into which .htaccess file do we insert the Rewrite
> statements? There are .htaccess files scattered all over the
> place but none in the Phorum top level.
Just create a new one in the directory you place the script.
> 2) Within topics.txt, you forget to include the step which
> discusses creating topics.html. The script itself is saved
> *AS* topics.html?
There ist nothing special. The topics.txt just generates the list with the links. You can take any html file and place the topics.txt (place <? at the beginning and ?> at the end :) ) in it. See the example at:
[www.manic-depressive.de]
The list is generated by the script.
> 3) To which directory does topics.html get uploaded?
that is your decision. Place it whereever you want.
Do you have any problems by using the script?
Re: make phorum spider friendly May 12, 2002 02:05PM |
Registered: 22 years ago Posts: 16 |
I'm sorry if this sounds dumb but I can't get it to work at all and I can't help but feel there's a missing piece to this puzzle. All I have is your topics.txt file and the information contained in these message threads.
I know I must have a .htaccess in the same directory with the Rewrite rules in it. I did that. I pasted the script from topics.txt into a file named topics.html and uploaded it to that same directory. Browsing for topics-0.html returned 404, not found. Browsing topics.html returned the raw code from the script. Is there anything else?
I'm on a Rackspace server just like phorum.org. mod_rewrite is installed. I know it's working because I use Rewrite rules extensively in httpd.conf <virtualhost> containers.
I have examined [www.manic-depressive.de]. All I see is the output from your script, not the script itself. Nor can I see the syntax you used when building topics.html. Having that would be extremely helpful. In fact, you might want to consider including a complete example of the html file in topics.txt.
I know I must have a .htaccess in the same directory with the Rewrite rules in it. I did that. I pasted the script from topics.txt into a file named topics.html and uploaded it to that same directory. Browsing for topics-0.html returned 404, not found. Browsing topics.html returned the raw code from the script. Is there anything else?
I'm on a Rackspace server just like phorum.org. mod_rewrite is installed. I know it's working because I use Rewrite rules extensively in httpd.conf <virtualhost> containers.
I have examined [www.manic-depressive.de]. All I see is the output from your script, not the script itself. Nor can I see the syntax you used when building topics.html. Having that would be extremely helpful. In fact, you might want to consider including a complete example of the html file in topics.txt.
Re: make phorum spider friendly May 12, 2002 02:16PM |
Re: make phorum spider friendly May 12, 2002 03:02PM |
Registered: 22 years ago Posts: 16 |
AH HA! The REAL confusion came from this statement from your first post in this thread:
4) create a file called topics.html (attention, not .php) and put the script into it.
It's really the other way around:
3.5) create a file called topics.php (attention, not .html) and put the script into it.
I have to run out for a few hours but I'll try it again when I get back. I'll let you know.
Mark
4) create a file called topics.html (attention, not .php) and put the script into it.
It's really the other way around:
3.5) create a file called topics.php (attention, not .html) and put the script into it.
I have to run out for a few hours but I'll try it again when I get back. I'll let you know.
Mark
Re: make phorum spider friendly May 12, 2002 07:31PM |
Registered: 22 years ago Posts: 16 |
IT WORKS!
[surfsafely.com]
Now I just need to finish off the page layout and link to it from the Phorum top menu so search engine spiders will find it.
Any chance we can get English translations for the German error codes? ;)
Mark
[surfsafely.com]
Now I just need to finish off the page layout and link to it from the Phorum top menu so search engine spiders will find it.
Any chance we can get English translations for the German error codes? ;)
Mark
Re: make phorum spider friendly May 13, 2002 02:36AM |
Hi,
> Any chance we can get English translations for the German
> error codes? ;)
yes :)
[...]
@mysql_connect($host,$user,$password)
or die("abort: no connection to '$host'");
// select database or die
@mysql_select_db($database)
or die("abort: database '$database' could not be selected"
."<br><br>MySQL says: ".mysql_error());
[...]
if ($start > 0) {
$newStart = ($start - $limit < 0) ? 0 : ($start-$limit);
echo "<a href=\"topics-" . $newStart . ".html\"><< back</a> ";
}
if ($start + $limit < $total) {
$newStart = $start + $limit;
echo "<a href=\"topics-" . $newStart . ".html\">forward >></a>";
}
[...]
> Any chance we can get English translations for the German
> error codes? ;)
yes :)
[...]
@mysql_connect($host,$user,$password)
or die("abort: no connection to '$host'");
// select database or die
@mysql_select_db($database)
or die("abort: database '$database' could not be selected"
."<br><br>MySQL says: ".mysql_error());
[...]
if ($start > 0) {
$newStart = ($start - $limit < 0) ? 0 : ($start-$limit);
echo "<a href=\"topics-" . $newStart . ".html\"><< back</a> ";
}
if ($start + $limit < $total) {
$newStart = $start + $limit;
echo "<a href=\"topics-" . $newStart . ".html\">forward >></a>";
}
[...]
Re: make phorum spider friendly March 18, 2003 02:05PM |
Registered: 19 years ago Posts: 4 |
Re: make phorum spider friendly March 20, 2003 09:55AM |
Registered: 19 years ago Posts: 4 |
Re: make phorum spider friendly June 26, 2003 12:51AM |
Registered: 22 years ago Posts: 13 |
Sorry, you do not have permission to post/reply in this forum.