GurujiBot/1.0

Obeyed Robots.txt

[mcneely]

72.20.109.xx GurujiBot/1.0 came through this morning and requested robots.txt before proceeding to attempt to collect home page from a site.

This is the 3rd time the bot has done this in roughly as many weeks so I'll be removing the 403 to see how it goes.

[Samizdata]

Guruji is an Indian search engine seeking market share in the same way Baidu dominates in China.

It seems well-behaved, and I allow it to crawl some of my sites that have appropriate content.

English is widely understood in India.

...

[dstiles]

Odd it's hosted in the USA considering the high-profile Indian internet. Not saying it's not a good engine (I don't know), just a bit odd.