Findshare bot

Found this Russian? bot hitting my logs today.

[grandma genie]

Hi,
Didn't see anything about this in a google search or search on this site, so am just passing this along for your info. I blocked it.

92.241.182.nn - - [04/Oct/2010:19:33:21 -0400] "GET /robots.txt HTTP/1.1" 200 1155 "-" "Mozilla/5.0 (compatible; FindShare bot/3.0; +http://www.findshare.ru)"
92.241.182.nn - - [04/Oct/2010:19:33:26 -0400] "GET / HTTP/1.1" 200 31375 "-" "Mozilla/5.0 (compatible; FindShare bot/3.0; +http://www.findshare.ru)"
92.241.182.nn - - [04/Oct/2010:19:33:30 -0400] "GET / HTTP/1.1" 200 31375 "-" "Mozilla/5.0 (compatible; FindShare bot/3.0; +http://www.findshare.ru)"

[Pfui]

That's a new one to me. Did you have it disallowed in robots.txt and it ignored same? Or are bots generally allowed to hit root and this one behaved itself?

(I don't like bots that simultaneously hit / and /robots.txt because Disallow includes root. Bots don't get one free bite, imho.)

[grandma genie]

No, it hit lots of files (I thought there were too many to list here) and I just found it today, so blocked it in htaccess. I always block visitors from certain countries, like Russia. My robots.txt file only includes files I don't want accessed by any bots or certain bots. I did not put FindShare in my robots.txt file because I didn't want to wait to see if it would obey it, I just wanted it to go away.

[grandma genie]

Now that you mention it, it appears that this bot did obey my robots.txt file. It did not access any of the files I disallowed in robots.txt, but I still blocked it just because of where it was coming from.

[grandma genie]

You know, Pfui, there is a website called findshare.com that claims to "Find the files from RapidShare, MegaUpload, Badongo, FileFront, SaveFile...! FindShare.com is a tool which allows users to find freely available files from all around the web! Maybe this bot is from that site.

[Pfui]

Dunno, sorry. But you know, if even you block hits from high-risk areas/ISPs/IPS, it's okay to allow access to robots.txt. Not only is it good policy/Netizenship/whatever, but often times bots will re-re-re-re-re-hit files when they're not specifically told "No" in robots.txt, because, well, that's what bots do, good and bad.

FWIW... Well-behaving 'visitors' of any kind from Russia (and certain other countries) are rare as hen's teeth on my sites. But if any bot asks for robots.txt, they get it. Non-major SEs get a dynamic one -- search WW for: dynamic robots.txt -- that discloses nothing but a blanket Disallow, but they still get it. And nothing else:)

[Dijkgraaf]

Project Honeypot

Geographic Location [Russia] Russia
Spider First Seen approximately 1 year, 10 months, 3 weeks ago
Spider Last Seen within 1 week
Spider Sightings 4,145 visit(s)
User-Agents seen with 3 user-agent(s)
Threat Rating 46

First Rule-Break On approximately 1 year, 3 months, 1 week ago
Last Rule-Break On within 1 year, 2 months, 3 weeks
Rule Breaks 6 web page navigation rule(s) broken by this IP

User Agent Strings
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
Mozilla/5.0 (compatible; FindShare bot/3.0; +http://www.findshare.ru)
Mozilla/5.0 (compatible; Tagoobot/3.0; +http://www.tagoo.ru)

[Pfui]

Hmm... Links to specific PHP pages would give us clues to (& more info about) IPs without visibly posting same. Got link?:)

[grandma genie]

I don't know about anyone else, but this bot did not request any of my php files. It was only hitting the html files. It basically went into each different directory and hit the opening html page in that directory, then went to the next one. It did not take any images.

[Pfui]

I guess I don't see where it did anything wrong.

[grandma genie]

Not yet. But these Russian, Chinese, Iranian, etc. visitors usually just sniff around at first, then, when you're not looking, whammmm! It's best to nip them in the bud.

[Dijkgraaf]

It will request PHP pages. The second page it requested from my site was the Project Honeypot page (which is a PHP one).