Welcome to WebmasterWorld Guest from 54.162.157.249

Forum Moderators: goodroi

Message Too Old, No Replies

asterias/2.0

Looks for streaming media files...

     
6:52 am on Jan 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Found a new one perusing my site a little bit ago and figured I'd pass it along, since it hasn't been mentioned here before.

http://www.singingfish.com/help/spider.html

206.253.222.*** - - [21/Jan/2004:13:21:57 -0800] "GET / HTTP/1.1" 200 20402 "-" "asterias/2.0"
206.253.222.*** - - [21/Jan/2004:13:21:58 -0800] "GET /robots.txt HTTP/1.1" 200 1524 "-" "asterias/2.0"

Reading thru these results [google.com] seems to indicate this critter has no business on my site.

I'm thinking:

User-agent: asterias
Disallow: /

Would not be as good as:

RewriteCond %{HTTP_USER_AGENT} asterias [NC,OR]

I believe I have that correct.

I don't much care if it asks for and obeys robots.txt or not, I simply don't have any files it'd need for that engine and can NOT see allowing it.

5:42 pm on Jan 23, 2004 (gmt 0)
9:52 pm on Jan 23, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The question was in the last statement, ( albiet ambiguously ):

I'm thinking:

User-agent: asterias 
Disallow: /

Would not be as good as:

RewriteCond %{HTTP_USER_AGENT} asterias [NC,OR]

I believe I have that correct?

Question mark added this time.

9:54 pm on Jan 23, 2004 (gmt 0)

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member



p - that's correct, especically if it's weird about not obeying your robots.txt. Their bot has been evil to a couple of our sites in the past.

Note: You may want to revisit this rule in 6 months. Singingfish could become a semi-major player. It's been getting a lot of hype.

If you don't have multimedia content, however, it's probably nothing to worry about.

10:07 pm on Jan 23, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There is nothing on my site they'd ever want and I'd just as soon ban them if for no other reason, than to save bandwidth. :)

Thanks.

11:43 pm on Feb 1, 2004 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Just had asterias; 2.0 hit my site. It did not obey robots.txt, hell... it didn't even request it.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month