asterias/2.0

Looks for streaming media files...

[pendanticist]

Found a new one perusing my site a little bit ago and figured I'd pass it along, since it hasn't been mentioned here before.

http://www.singingfish.com/help/spider.html

206.253.222.*** - - [21/Jan/2004:13:21:57 -0800] "GET / HTTP/1.1" 200 20402 "-" "asterias/2.0"
206.253.222.*** - - [21/Jan/2004:13:21:58 -0800] "GET /robots.txt HTTP/1.1" 200 1524 "-" "asterias/2.0"

Reading thru these results [google.com] seems to indicate this critter has no business on my site.

I'm thinking:

User-agent: asterias
Disallow: /

Would not be as good as:

RewriteCond %{HTTP_USER_AGENT} asterias [NC,OR]

I believe I have that correct.

I don't much care if it asks for and obeys robots.txt or not, I simply don't have any files it'd need for that engine and can NOT see allowing it.

[bcolflesh]

[webmasterworld.com...]

[webmasterworld.com...]

[pendanticist]

The question was in the last statement, ( albiet ambiguously ):

I'm thinking:

User-agent: asterias 
Disallow: /

Would not be as good as:

RewriteCond %{HTTP_USER_AGENT} asterias [NC,OR]

I believe I have that correct?

Question mark added this time.

[bakedjake]

p - that's correct, especically if it's weird about not obeying your robots.txt. Their bot has been evil to a couple of our sites in the past.

Note: You may want to revisit this rule in 6 months. Singingfish could become a semi-major player. It's been getting a lot of hype.

If you don't have multimedia content, however, it's probably nothing to worry about.

[pendanticist]

There is nothing on my site they'd ever want and I'd just as soon ban them if for no other reason, than to save bandwidth. :)

Thanks.

[keyplyr]

Just had asterias; 2.0 hit my site. It did not obey robots.txt, hell... it didn't even request it.