Forum Moderators: open

Message Too Old, No Replies

SEOdiver

         

keyplyr

10:06 pm on Jan 6, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: Mozilla/5.0 (compatible; SEOdiver/1.0; +http://www.seodiver.com/bot)
Protocol: HTTP/1.1
Robots.txt: No
Host: hetzner.de
144.76.0.0 - 144.76.255.255
144.76.0.0/16

Another set of SEO tools ironically, boasting of robots.txt support that did not happen.

deuces

1:32 am on Jan 10, 2016 (gmt 0)

10+ Year Member Top Contributors Of The Month



Yup, coincidentally, I started getting these yesterday.

iamlost

2:29 pm on Jan 10, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Took a look back through logfiles: first sighting was on a UK site in August 2014, first NA site: July 2015. Automagically binned first to last.

I presume Alan chose the name 'seodiver' as a branding exercise...however, shouldn't he be aware that one of the first listed user-agent block terms for a very long time has been 'seo'?

keyplyr

6:51 am on Jan 16, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Then you need to respect the rights of web site owners by requesting robots.txt and obeying that standard. You also need to be truthful about what you do with the data your bot scrapes from our web sites without our permission.

Just saying that your bot supports web standards, but then ignores robots.txt and nocache directives is not ethical.

Short of that, your "tool" remains blocked.

iamlost

2:21 pm on Jan 16, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




Then you need to respect the rights of web site owners by requesting robots.txt and obeying that standard. You also need to be truthful about what you do with the data your bot scrapes from our web sites without our permission.

Absolutely.
A lot of the folks out there don't walk their talk. Idjits.

keyplyr

10:18 pm on Jan 16, 2016 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@iamlost - of course when I said "you need to" above, I was speaking to the bot owner. He had posted some spammy hyperbole about how valuable his "tool" was (removed by admin.)