Forum Moderators: phranque

Message Too Old, No Replies

.htaccess ban list

help please

         

philipvs

6:57 pm on Apr 19, 2007 (gmt 0)

10+ Year Member



In the past here was created topic [webmasterworld.com...]

Tell please how you see what bots (like: ^EmailSiphon, ^Zeus.*Webster etc.) visit your site?
Where I can find info about bots that visit my site, so I can ban them?
Sorry for my bad English.
Thank you.

jdMorgan

10:46 pm on Apr 19, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Generally, your host provides server access log files, which you can access with HTTP or through FTP, to see every request that your server receives.

The entries look like this:


72.30.177.225 - - [19/Apr/2007:03:13:21 -0400] "GET /robots.txt HTTP/1.0" 200 6553 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

In this case, the request was from Yahoo's search engine spider, named "Slurp". The IP address belongs to Intomi, a company that Yahoo purchased several years ago.

Jim