| Welcome to WebmasterWorld Guest from 18.104.22.168 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
| 2:08 am on Jun 30, 2000 (gmt 0)|
User Agent: Mozilla/4.07 [en]
Would they have a spider???
| 2:33 am on Jun 30, 2000 (gmt 0)|
The whois says it's SNAP just like you said, did it ask for robots.txt? Could be a link checker.
| 3:00 am on Jul 8, 2000 (gmt 0)|
SNAP is now spidering upon LiveDirectory Submissions...
We've been showing it's crawler, pages that rank well in Inktomi, and have achieved some nice rankings in LiveDirectory..
| 7:58 am on Jul 9, 2000 (gmt 0)|
Hey Redzone, thanks for the info. That's what I was theorizing. Do you have the User Agent/IP of that by any chance?
Thanks for any help.
| 3:31 pm on Jul 9, 2000 (gmt 0)|
Here is what we see coming in for the crawl. We have a dedicated person for SNAP LD submissions, and SNAP usually crawls within 8 hours of the submission.
UA - Mozilla/4.07 [en]
IP - 206.132.166
So that matches what you've seen..
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved