Forum Moderators: open

Message Too Old, No Replies

Which spider is this?

Unknown robot (identified by 'crawl')

         

zoobie

2:49 pm on Feb 16, 2004 (gmt 0)

10+ Year Member



I was wondering which spider visited my site? Can't seem to find out by searching. Thanks
Unknown robot (identified by 'crawl')

Hope

9:47 pm on Feb 17, 2004 (gmt 0)

10+ Year Member



What is the IP of the spider?

zoobie

3:16 am on Feb 18, 2004 (gmt 0)

10+ Year Member



Dunno...I looked and found mamma and ix quick meta search has me listed...and now google. Where would I look? Thanks

Hope

4:07 pm on Feb 19, 2004 (gmt 0)

10+ Year Member



You will need to open up your raw logs. Pinpoint the day the spider hit. Then read through your logs until you find it. There will be an ip that will better help someone identify your spider.

Google's spider is Googlebot and any decent log analyzer should tell you that. A few other bots to look out for:

Slurp (Inktomi)
Scooter (Alta Vista)
Fast Webcrawler (Fast/AllTheWeb)
Yahoo-MMCrawler (Yahoo)
Yahoo! Slurp (New Yahoo/Inktomi spider name)

webnewton

10:30 am on Feb 21, 2004 (gmt 0)

10+ Year Member



Log analyzers keep a track of just a few known spiders. There are many more spiders crawling the web 24 hours. It may be one of the software searching for the link partmners

zoobie

6:31 pm on Feb 21, 2004 (gmt 0)

10+ Year Member



Alright...Thanks
I looked in the log but it just said the same thing.