Forum Moderators: DixonJones

Message Too Old, No Replies

normal web page acts as spider

         

helenp

1:08 am on Mar 19, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hi I donīt know if this is where to post this,
but often seen in stats, that suddenly some webpage I ever seen, and without links to my site, are an referer for several visitors.

Today had 56 visits in 2 hours from an german webpage.
Downloaded the raw file and seen itīs really visits 56 diferents pages of mine, as if where an spider.
How and why can this happens?

This is an exampel from my raw file:
211.152.14.98 - - [18/Mar/2004:10:22:46 -0500] "GET /sitemap.htm HTTP/1.1" 404 1766 "http://website.html" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.98 - - [18/Mar/2004:10:22:53 -0500] "GET /mysite_1.htm HTTP/1.1" 404 1766 "http://website.html" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.98 - - [18/Mar/2004:10:22:54 -0500] "GET /mysite_2.htm HTTP/1.1" 404 1766 "http://website.html" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98"
211.152.14.98 - - [18/Mar/2004:10:22:58 -0500] "GET

All from same Ip.....and always diferent pages of mine.

jdMorgan

1:16 am on Mar 19, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



helenp,

It's simply a spider written to use a false referrer and a fake user-agent name.

Jim

bull

6:20 am on Mar 19, 2004 (gmt 0)

10+ Year Member



I'm wondering if MSIE 5.00 is a valid user-agent at all? (At least FlashGet (download program) uses it).

Additionally, the ending ')' is missing here.

helenp

10:06 am on Mar 19, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thanks,
Helen