Welcome to WebmasterWorld Guest from 54.242.53.253

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

Scooter/2.0 G.R.A.B. V1.1.0

     
2:22 pm on Jan 18, 2000 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 21, 1999
posts:370
votes: 0


Hi Guys, I have been heavily crawled by this A.V agent. Any info would be great.

Thanks,
Pete

4:21 am on Jan 19, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38063
votes: 13


What is it specifically you need to know? Scooter is Altavista's standard spider. Getting crawled by them is a major accomplishment these days (congrats). Or is there some problem with the spidering?
10:43 am on Jan 19, 2000 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 21, 1999
posts:370
votes: 0


Hi Brett,

Apologies for any ambiguity. I have been crawled by a number of A.V's spiders in the past. This is the first time that I have been crawled by the above agent (2.0 G.R.A.B. V1.1.0). As yet , none of the pages have been included. What would have sparked the spider to crawl this site. The site is new and I havent submitted any pages to A.V yet. As to my original question, what is this spiders purpose: crawl & include pages in the A.V database / record the number of links for link popularity purposes / collect images for A.V's image database e.t.c.

I hope this is clearer.
Thanks,
pete

RealSpices

12:29 am on Jan 20, 2000 (gmt 0)

Inactive Member
Account Expired

 
 


That one has been over my site like gangbusters since AV unstuck. A little wierd though, kinda like a little kid just let out to play! Scooter looks to be very excited, looking for pages like index.htmindex2.htm (yup two pages run together)

hbhb

3:55 pm on Feb 1, 2000 (gmt 0)

Inactive Member
Account Expired

 
 


I have been hit by scooter several times in the past few weeks but each time they have only requested my robots.txt (which I don't have) and then took off. do you guys have any clue about what is going on? it did not bother to look at any of my other pages.
hb
5:04 pm on Feb 1, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38063
votes: 13


Are you looking at raw log files? If so, look for HEAD requests by scooter. AV checks the document age by just doing HEAD checks on a document. If the doc hasn't updated since the last time it was there, it moves on.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members