Spam Hurdles Module (CAPTCHA's and other anti-spam tools)
Posted by Maurice Makaay
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 03, 2006 06:45AM |
Registered: 21 years ago Posts: 687 |
This is a fantastic piece of work, thank you Maurice.
It was such a pleasant surprise to install a relatively complex module without having to even make a template change - brilliant!
/\dam
--
My notable Phorum sites:
Movie Deaths Database - "review comments" system mostly powered by Phorum
Learn Chinese! - integrated forum quiz
It was such a pleasant surprise to install a relatively complex module without having to even make a template change - brilliant!
/\dam
--
My notable Phorum sites:
Movie Deaths Database - "review comments" system mostly powered by Phorum
Learn Chinese! - integrated forum quiz
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 03, 2006 08:06AM |
Admin Registered: 20 years ago Posts: 8,532 |
Not having to do any hacks on templates and code is what defines a module for me, so I always try to make my modules work as a simple to use drop-in block of code. I'm glad you like it. I hope it will keep the spammers away from your forums =)
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 03, 2006 04:36PM |
Registered: 21 years ago Posts: 687 |
> I hope it will keep the spammers away from your forums =)
I upgraded the forums on www.moviedeaths.com last night, but stopped to have dinner before adding your anti-spam module (I figured the one I coded would be unlikely to work on the latest version without a rewrite). 50+ automated spams were posted within 45 minutes :(
On other Phorum sites I run, the Apache webserver has even been taken down due to the volume of automated spam attempts - so even blocking the posts is sometimes not enough to protect a site unfortunately.
For this reason I believe all forum software should now ship with anti-spam measures enabled by default. This would at least mean that the potential gains from running automated attacks would be lessened. It is no longer the case that vulnerable sites don't affect us, as they indirectly increase the chance our own sites will be targeted. Scripts don't seem to care that their spam posts don't get through*, they continue their batch posting for days at a time regardless, which obviously causes significant load on our servers.
/\dam
* quite why they don't care is a mystery to me, as it is surely a waste of a spammer's resources to continually POST to a form with a CAPTCHA, but my own logs show that time and time again this is not the case, they just post regardlesss.
--
My notable Phorum sites:
Movie Deaths Database - "review comments" system mostly powered by Phorum
Learn Chinese! - integrated forum quiz
I upgraded the forums on www.moviedeaths.com last night, but stopped to have dinner before adding your anti-spam module (I figured the one I coded would be unlikely to work on the latest version without a rewrite). 50+ automated spams were posted within 45 minutes :(
On other Phorum sites I run, the Apache webserver has even been taken down due to the volume of automated spam attempts - so even blocking the posts is sometimes not enough to protect a site unfortunately.
For this reason I believe all forum software should now ship with anti-spam measures enabled by default. This would at least mean that the potential gains from running automated attacks would be lessened. It is no longer the case that vulnerable sites don't affect us, as they indirectly increase the chance our own sites will be targeted. Scripts don't seem to care that their spam posts don't get through*, they continue their batch posting for days at a time regardless, which obviously causes significant load on our servers.
/\dam
* quite why they don't care is a mystery to me, as it is surely a waste of a spammer's resources to continually POST to a form with a CAPTCHA, but my own logs show that time and time again this is not the case, they just post regardlesss.
--
My notable Phorum sites:
Movie Deaths Database - "review comments" system mostly powered by Phorum
Learn Chinese! - integrated forum quiz
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 03, 2006 05:17PM |
Admin Registered: 20 years ago Posts: 8,532 |
Especially the fact that you let through a bunch of posts might cause some extra spam attempts in the upcoming time. What I have seen on my company's systems, is that after a successful post, spammers will simply cache the succeeded post data and directly post that data on new attempts. So they won't even bother to load the posting form anymore, since they already know what data has to be posted. This kind of attacks is what I stop by using a posting key that can be used only once. So even if a first post would be successful (possibly initiated manually by the spammer), then successive posts would be blocked.
But I'm glad that you get this amount of spam. This makes you a good test case for my module ;-) I never had spam on my own site, so I have no good test situation, which makes this module a bit of a theoretical excercise for me.
Ship Phorum with anti-spam tools or not?
I agree with you that spam is getting worse by the day and that we might need to ship anti-spam tools with Phorum.
On the other hand, there's the biological mechanism of diversity. A species grows weak if there is not enough diversity. So when all Phorum installations will be the same in regards to the used anti-spam measures, then it's easier for spammers to find a route to pass them. Writing a script that handles one site can be used to attack all other sites as well. This situation would only be stimulated by shipping the Spam Hurdles module (or whatever anti-spam tool) with Phorum.
Rapid evolution would be nature's rescue in such situation and the same goes for Phorum anti-spam tools. By regularly adding and updating spam blocks, the strength of the tools is improved. But this also means that regular updates of the spamtools might be required to keep the shields up at a good level. This contradicts with shipping the tools as a part of Phorum, since Phorum updates aren't that regular and admins tend to keep running the version of Phorum they installed. Installing and upgrading anti-spam tools separately is probably better in this case. A combination (ship and separate upgrades) is possible too of course.
These are some of my personal considerations about shipping anti-spam tools as a default option with Phorum.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
But I'm glad that you get this amount of spam. This makes you a good test case for my module ;-) I never had spam on my own site, so I have no good test situation, which makes this module a bit of a theoretical excercise for me.
Ship Phorum with anti-spam tools or not?
I agree with you that spam is getting worse by the day and that we might need to ship anti-spam tools with Phorum.
On the other hand, there's the biological mechanism of diversity. A species grows weak if there is not enough diversity. So when all Phorum installations will be the same in regards to the used anti-spam measures, then it's easier for spammers to find a route to pass them. Writing a script that handles one site can be used to attack all other sites as well. This situation would only be stimulated by shipping the Spam Hurdles module (or whatever anti-spam tool) with Phorum.
Rapid evolution would be nature's rescue in such situation and the same goes for Phorum anti-spam tools. By regularly adding and updating spam blocks, the strength of the tools is improved. But this also means that regular updates of the spamtools might be required to keep the shields up at a good level. This contradicts with shipping the tools as a part of Phorum, since Phorum updates aren't that regular and admins tend to keep running the version of Phorum they installed. Installing and upgrading anti-spam tools separately is probably better in this case. A combination (ship and separate upgrades) is possible too of course.
These are some of my personal considerations about shipping anti-spam tools as a default option with Phorum.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 03, 2006 06:16PM |
Registered: 21 years ago Posts: 687 |
>spammers will simply cache the succeeded post data and directly
>post that data on new attempts.
I see similar behaviour, but the caching sometimes (in the worst case) gets farmed out to a small army of zombie machines, which obviously means that every incoming attack originates from a different IP address. Even worse, the post requests manage to tie up an Apache connection for a far longer time than a normal browser - this is what ultimately takes my servers down.
In these situations I have had to rename my Phorum post.php form completely (fortunately easy to do with the codebase) and use a .htaccess file to block requests to the old form.
One feature that would be very useful would be to be able to spot spamming IP addresses and add them temporarily to a .htaccess file to deny them completely. This causes less load than blocking the attempts via PHP.
Regarding keeping anti-spam measures as diverse as possible, I believe your current module has already made a good start along that path.
/\dam
--
My notable Phorum sites:
Movie Deaths Database - "review comments" system mostly powered by Phorum
Learn Chinese! - integrated forum quiz
>post that data on new attempts.
I see similar behaviour, but the caching sometimes (in the worst case) gets farmed out to a small army of zombie machines, which obviously means that every incoming attack originates from a different IP address. Even worse, the post requests manage to tie up an Apache connection for a far longer time than a normal browser - this is what ultimately takes my servers down.
In these situations I have had to rename my Phorum post.php form completely (fortunately easy to do with the codebase) and use a .htaccess file to block requests to the old form.
One feature that would be very useful would be to be able to spot spamming IP addresses and add them temporarily to a .htaccess file to deny them completely. This causes less load than blocking the attempts via PHP.
Regarding keeping anti-spam measures as diverse as possible, I believe your current module has already made a good start along that path.
/\dam
--
My notable Phorum sites:
Movie Deaths Database - "review comments" system mostly powered by Phorum
Learn Chinese! - integrated forum quiz
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 03, 2006 06:32PM |
Admin Registered: 20 years ago Posts: 8,532 |
Tip: If there are too many spam clients that use up your Apache connections, then it might be an idea to disable keep-alive connections temporarily. I've had to do that on several occasions, where zombies were ddossing our servers.
Adding entries dynamically to a blocklist is a good idea. I wouldn't choose for writing them to a .htaccess file directly though. I'd say that a database table has to be maintained on spammers. If a certain IP-address hits the spamblocks a couple of times, then that address can be blacklisted for a certain amount of time (let's say half an hour to prevent too much problems for dynamic IP users). This can all be done from within Phorum. Using the "scheduled" hook (belongs to script.php), it would be possible to export the blocklist to various targets. One would be .htaccess, but other options are firewall rules or even access list in network routers. This allows for maximum flexibility.
I'll certainly keep the idea in mind for a next update of the mod. Thanks for the idea!
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Adding entries dynamically to a blocklist is a good idea. I wouldn't choose for writing them to a .htaccess file directly though. I'd say that a database table has to be maintained on spammers. If a certain IP-address hits the spamblocks a couple of times, then that address can be blacklisted for a certain amount of time (let's say half an hour to prevent too much problems for dynamic IP users). This can all be done from within Phorum. Using the "scheduled" hook (belongs to script.php), it would be possible to export the blocklist to various targets. One would be .htaccess, but other options are firewall rules or even access list in network routers. This allows for maximum flexibility.
I'll certainly keep the idea in mind for a next update of the mod. Thanks for the idea!
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 04, 2006 04:57AM |
Registered: 18 years ago Posts: 6 |
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 04, 2006 04:58AM |
Admin Registered: 18 years ago Posts: 746 |
Hi Sheik,
I can observe on my server automated attacks against the forum an each contact form. Each time I recognize an attack I check the IP to get the responsable provider. If it is a provider in Western Europe, I write a email to abuse. Normally this helps. If it is outside, I put the complete IP-range of this provider on my ban list.
For monthes I didn't get any automated spam to our forum. For our contact forms I get actually once a month an attempt.
Regards
Oliver
Using Phorum since 7/2000: forum.langzeittest.de (actual version 5.2.23)
Modules "Made in Germany" for version 5.2: Author_as_Sender, CarCost, Close_Topic, Conceal_Message_Timestamp,
Format_Email, Index_Structure, Mailing_List, Pervasive_Forum, Spritmonitor, Terms_of_Service and German_Language_Files_Package.
Quote
sheik
50+ automated spams were posted within 45 minutes
I can observe on my server automated attacks against the forum an each contact form. Each time I recognize an attack I check the IP to get the responsable provider. If it is a provider in Western Europe, I write a email to abuse. Normally this helps. If it is outside, I put the complete IP-range of this provider on my ban list.
For monthes I didn't get any automated spam to our forum. For our contact forms I get actually once a month an attempt.
Regards
Oliver
Using Phorum since 7/2000: forum.langzeittest.de (actual version 5.2.23)
Modules "Made in Germany" for version 5.2: Author_as_Sender, CarCost, Close_Topic, Conceal_Message_Timestamp,
Format_Email, Index_Structure, Mailing_List, Pervasive_Forum, Spritmonitor, Terms_of_Service and German_Language_Files_Package.
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 04, 2006 05:19AM |
Admin Registered: 20 years ago Posts: 8,532 |
Quote
martin72
Phorum 5.1.16 + "spamhurdles" mod 1.0.7 doesn't work at all. I delete about 200 spam msgs a day.
Please, be specific. The worst comment that you can give a programmer is "doesn't work". Programmers can do nothing with those.
Do you have problems running the module, what are your settings, are those 200 posts coming through after installing and running spamhurdles, did you run earlier versions of spamhurdles too or is this the first one, if you enabled the captcha then do you see the captcha when posting, if you enabled the option to block too quickly posted messages then do you see the 5 second count-down on the post message button?
That's the kind of information that is useful, so please elaborate on your exact problem.
Maurice Makaay
Phorum Development Team
my blog linkedin profile secret sauce
Re: Spam Hurdles Module (CAPTCHA's and other anti-spam tools) October 04, 2006 05:28AM |
Registered: 18 years ago Posts: 35 |
Hi,
I have been running Spam Hurdles quite some time now and not one spam message (or account) appeared. But I we have another problem using it.
Sometimes when going into my Private Messages box to read a new PM and then clicking the Answer button resolves in an error message by Spam Hurdles (something like: can't determine wether we are writing a new message or editing one [or something]).
I can't reproduce that error right now but several people got it independently. I also experienced that after that error occured I wasn't able to answer any of my PMs nor could I write a completely new one. Everything resolved into the mentioned error.
Any idea why this is?
Sorry to cause trouble again,
Netwalker.
I have been running Spam Hurdles quite some time now and not one spam message (or account) appeared. But I we have another problem using it.
Sometimes when going into my Private Messages box to read a new PM and then clicking the Answer button resolves in an error message by Spam Hurdles (something like: can't determine wether we are writing a new message or editing one [or something]).
I can't reproduce that error right now but several people got it independently. I also experienced that after that error occured I wasn't able to answer any of my PMs nor could I write a completely new one. Everything resolved into the mentioned error.
Any idea why this is?
Sorry to cause trouble again,
Netwalker.
Sorry, only registered users may post in this forum.