jonathanleger - 6:47 am on Feb 25, 2012 (gmt 0)

Yeah, it's me.

Sorry about the lack of a user-agent. It was set in the INI but wasn't getting passed to CURL for some reason. I didn't realize that until somebody put a support ticket in asking what was up. Fortunately that was caught and corrected very early.

As far as what the purpose of the crawling is, think SEOMoz.org with a different focus.

Of course, the crawlers respect robots.txt and will abide if you don't want them around.