homepage Welcome to WebmasterWorld Guest from 54.161.247.22
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

  posting off  
slurped to death
oilman




msg:19059
 10:38 pm on Sep 18, 2000 (gmt 0)

ok - I got slurped 3 times today from 3 different IPs. Those didn't bother me - they were all known but then this showed up and only grabbed my main page:

ip: 209.185.208.29
reverse DNS: ALLISONB
from: Exodus Communications
Time: 6 min
Browser: Mozilla/4.0 (compatible; MSIE 5.01; Window NT 5.0)
refering url: www.some.real.url

Is this someone from Exodus legitimatly checking out our product by following the url from somewhere or is this an arachnoid that I don't have in my list?

Also saw this little guy: 209.185.240.250 - didn't have time to dig into it yet but I know it's not on my list.

 

Kamikaze




msg:19060
 1:23 am on Sep 19, 2000 (gmt 0)

Oilman- Would you happen to know if it grabbed your robots.txt file?

oilman




msg:19061
 1:32 am on Sep 19, 2000 (gmt 0)

The first one sure didn't. I was pretty disappointed that it didn't. ;) That makes life so much easier.

Can't say for sure on that second IP though.

Gorufu




msg:19062
 12:43 pm on Sep 21, 2000 (gmt 0)

Hi oilman,

209.185.208.29 appears to be a legitimate visitor, especially if there was a referring URL.

209.185.240.250 www.law3.hotmail.com

BTW: I did a little bit of digging and found that
Exodus Communications is the NetBlock maintainer for
directhit.com
hotmail.com
hotbot.com
lycos.com
iwon.com
alltheweb.com
Inktomi (most Slurp spidering servers)
inktomi.com (NS3.INKTOMI.COM NS4.INKTOMI.COM)

Doing a traceroute all of the above domains go through
exodus-bos1-oc3.customer.ALTER.NET (157.130.15.202)
then different exodus.net servers before their final destination

oilman




msg:19063
 3:13 pm on Sep 21, 2000 (gmt 0)

Thanks Gorufu - that's what I figured as well. Welcome to the forums.

On another note: Anybody know what the y400 spider does for INK? She has been hitting me regularly for the past week at least 3 times a day. Pulls robots.txt and default.asp. Take about 6 minutes to do it then leaves.

oilman




msg:19064
 7:58 pm on Sep 22, 2000 (gmt 0)

Here's my log for y400 for the last 2 days:
anybody else seeing this kind of activity?

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-20 09:30:37 - user created
2000-09-20 09:30:37 /robots.txt
2000-09-20 09:30:40 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-20 15:31:21 - user created
2000-09-20 15:31:21 /robots.txt
2000-09-20 15:31:22 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-21 01:33:48 - user created
2000-09-21 01:33:48 /robots.txt
2000-09-21 01:33:58 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-21 11:21:38 - user created
2000-09-21 11:21:38 /robots.txt
2000-09-21 11:21:51 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-21 13:31:30 - user created
2000-09-21 13:31:30 /robots.txt
2000-09-21 13:31:31 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-21 15:33:14 - user created
2000-09-21 15:33:14 /robots.txt
2000-09-21 15:33:19 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-21 16:40:25 - user created
2000-09-21 16:40:25 /robots.txt
2000-09-21 16:40:25 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-21 21:29:22 - user created
2000-09-21 21:29:22 /robots.txt
2000-09-21 21:29:23 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-22 03:29:06 - user created
2000-09-22 03:29:06 /robots.txt
2000-09-22 03:29:12 /default.asp

* User Session 209.185.141.226 - Exodus Commnications Inc. (CA,US)
2000-09-22 09:29:58 - user created
2000-09-22 09:29:58 /robots.txt
2000-09-22 09:30:00 /default.asp

drbill




msg:19065
 1:15 pm on Sep 25, 2000 (gmt 0)

Oilman,

Slurp has been living on my server for the last 3 weeks.

Hitting hard but I see nothing in the way of traffic from any of the Ink Portals.

Brett_Tabke




msg:19066
 1:34 pm on Sep 25, 2000 (gmt 0)

Gorufu, thank you - that explains alot. I didn't know just what it was Exodus did, but knew many spiders sure ran through their system. Not a whole lot about them on the net.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved