seth_wilde
msg:26497 | 9:07 pm on Sep 5, 2000 (gmt 0) |
I'll have to look through some logs to verify (which I haven't had time to do yet), but as far as I know they're still doing the insta spider on all submissions to check for 404's, robot.txt, etc... What have you noticed?
|
redzone
msg:26498 | 2:04 pm on Sep 6, 2000 (gmt 0) |
Brett: We're seeing instant crawling on everything we've submitted over the last week. Approx. 800 URL's per day, across 200 domains.
|
Brett_Tabke
msg:26499 | 2:58 pm on Sep 6, 2000 (gmt 0) |
No I'm not seeing it visit on all submissions now (hand). I did just try some and they all had visits from:
128.177.243.86 bigip-nat.sv.alta-vista.net Scooter/1.0 But that wasn't happening on Sunday and Monday. I will keep a very close eye on it. It isn't a domain thing or anything like that - these were all new domains that were never indexed. For those watching turn around times. Just submitted a new one from:
[searchengineworld.com] (let me know when you find it in a search ;)
|
drbill
msg:26500 | 4:31 am on Sep 7, 2000 (gmt 0) |
Brett, I seem to be getting hit from the following.
Host:128.177.243.86
Agent:Scooter/1.0 and I am getting listed within 4 days on these submissions. Anybody else seeing this?
|
|
