asterias/2.0

So.........Does this bot always snoop for security holes?

[mcneely]

asterias/2.0

Not a very nice bot I'm afraid

Poked around for media files on a site that doesn't contain them, asked for the index twice, did look at the robots txt but completely ignored meta protected pages and made 4 requests for server root

(fish02.search.aol.com)source

So, what about it guys.............is this bot a slob or what?

[wilderness]

http//search.singingfish.com/sfw/home.jsp

(add the missing : )

I get some pests from this backbone.
My sites have neither audio or video. It's an easy choice for me.

was denied off a previous SetEnvIf line.

206.253.222.233 - - [20/Jan/2004:19:50:40 -0800] "GET / HTTP/1.1" 403 - "-"
"asterias/2.0"
Internap Network Services INTERNAP-SEA (NET-206-253-192-0-1)
206.253.192.0 - 206.253.223.255
Singingfish.com PNAP-SEA-SNGFSH-RM-02 (NET-206-253-222-224-1)
206.253.222.224 - 206.253.222.255

[Lord Majestic]

but completely ignored meta protected pages and made 4 requests for server root

What meta protection are you referring to?

[mcneely]

I have some pages scattered around that still have the Robots noindex tags in them

[Lord Majestic]

I have some pages scattered around that still have the Robots noindex tags in them

Were they indexed or were they merely crawled? I assume you aware that a crawler can't see that tag until the page is crawled, and then its up to indexer to honour this meta tag or not -- the page can still get crawled.

The point I am making here is that "completely ignored meta protected pages" makes no sense insofar as crawler is concerned.

[mcneely]

You are 100% right

By the time I had finished looking at all of the nosing that this insidious little bot did, I was pretty ticked off.

It only stands to reason that these misc pages wouldn't be indexed.

I went ahead and gave the old boy a few perfectly good 403's to chew on.