lucy24 - 1:54 am on May 6, 2011 (gmt 0)
OK, so you don't want to hear about how many detours it took me to arrive here:
(3 years ago, referring to robots.txt)
(1 1/2 years ago, referring to sitemaps)
Give it another 4 1/2 years and maybe google will admit that This Is Stupid ;)
I don't regret the detour, because it left me with this deathless quote:
these [i.e. robots.txt and sitemap.whatever] are urls and search engines index urls
Think I'll pin that to the bathroom wall.
Besides, it gave me something to do while waiting for the text editor to get bored with holding its breath and turning blue. When you neglect to Save before embarking on an ill-advised RegEx, the text editor holds all the cards.