Forum Moderators: goodroi

Message Too Old, No Replies

How to deal with a robots.txt hack attack

Guidelines to mitigate the damage from an attack

         

Whitey

12:59 am on Oct 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



A few folks have reported server attacks which have left owners with robots.txt across their sites. The consequence is a very unwelcome disruption with a 180 exclusion period on Google, probably leaving the site owners with little income.

[webmasterworld.com...]

[webmasterworld.com...]

Has anyone else experience this, and are there some guidelines to better manage both prevention and an urgent reinclusion with the search engines

goodroi

12:28 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Dealing with a hackers attack on your system is not really a robots.txt issue. It is more a general security issue of keeping system up to date with security upgrades. If a hacker compromises your system they can take you down in a number of ways. The silver lining is that hacking computers is illegal and you can turn to the police.

In general, people should realize the importance of robots.txt. Most critical robots.txt mistakes that I have come across are made out of ignorance. For example there is a large goverment site that currently is blocking Yahoo and MSN but allowing Google. When they contacted me for help with another issue, I pointed this out and no one in a decision making position knew about it.

Keep a complete back-up of your website including your roobts.txt and regualrly review it to make sure no one changes it without you being told.

Whitey

10:17 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



For example there is a large goverment site that currently is blocking Yahoo and MSN but allowing Google. ..... I pointed this out and no one in a decision making position knew about it.

This is scary, how many good sites just sit there with problems on them and no solutions. You can read about them all over these forums - and the people here are much more switched on than the average siteowner.

Is there a way to re instate pages within the 180 day exclusion period on Google?

In my mind this is another example of how webmastercentral could be improved greatly with better access to notifications and remedial steps. This is a clear case of "error" which is akin to "fixing" after a hack attack.