Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: bakedjake
But, they recently stated they were seeing crawling on new submissions to Infoseek within 24 hours.
We are not experiencing the same response.
Anybody else seeing crawling within 24 on "new" URL's being submitted?
I submitted a domain yesterday to go.com and shortly afterward saw the spider show up:
126.96.36.199 - - [11/Jul/2000:21:34:55 -0400] "GET /robots.txt "InfoSeek Sidewinder/0.9"
188.8.131.52 - - [11/Jul/2000:21:34:56 -0400] "GET / HTTP/1.0" "InfoSeek Sidewinder/0.9"
Yesterday also saw the spider show up on a domain I had submitted almost exactly a month before. However, after closer study, I think the spider is following links of sites submitted about 1 month ago to get there and the timing is coincidental. Also seeing the spider on domains submitted 1-1 1/2 months ago that were submitted by hand, root domain only, to go.com.