216.88.0.0-216.91.255.255

Forum Moderators: open

Message Too Old, No Replies

216.88.0.0-216.91.255.255

Savvis Communications - why spidering?

trillianjedi

1:53 pm on Jul 28, 2003 (gmt 0)

Are these guys looking to build a search engine or something?

Any idea why they are spidering?

bakedjake

1:59 pm on Jul 28, 2003 (gmt 0)

That's /22 of IP addresses.

SAVVIS is a tier-one backbone provider. It sounds like it's been assigned to an organization under SAVVIS, and they haven't SWIPed (entered it into ARIN's DB) yet.

trillianjedi

2:15 pm on Jul 28, 2003 (gmt 0)

OK, thanks.

Well whoever owns this one, it's not obeying robots.txt so just got banned.

jazzguy

3:37 pm on Jul 28, 2003 (gmt 0)

What user-agent are they using? Zyborg is crawling from a Savvis IP and got itself automatically banned by ignoring robots.txt.

See: ZyBorg/1.0 violates robots.txt [webmasterworld.com]

trillianjedi

3:39 pm on Jul 28, 2003 (gmt 0)

Thanks for the tip-off.

It was indeed ZyBorg.

How rude of them.

claus

5:55 pm on Jul 28, 2003 (gmt 0)

<error>

<sorry>