SlobberDawg

Well, okay, it's ours

[mcneely]

Most of you here might not ever see this because we mostly only use it for link audits of some of our many websites and client sites

Mozilla/4.5_(compatible;_WebShoppeHosting/_SlobberDawg5.01)
Only comes from these two ip's 69.145.2.161 & 69.145.11.58 and that's it.

Development has been considering putting together an index but can't decide on just what type of index to build

[bull]

Hope you have implemented robots.txt support! ;)

[mcneely]

Robots.txt is a requirement with this.

I don't know about any of you other mates, but chasing rude robots around all day is just not my cup of tea.

I would much rather be doing other things with my time.

It would be a great day when everyone would be respectfull enough to require robots.txt support for their bots.

Not having robots.txt support configured is just "rude"