Forum Moderators: open
Not one hit on the robots.txt fileHuh, that's odd. On my site they do ask for robots.txt, and were authorized after demonstrating compliance. (Unfortunately I don't keep careful records of this point, so I don't know how long it's been. But it must be pretty recent; for much of last year I only see them requesting robots.txt.) If not authorized, they would be blocked on header grounds.