Allow: ?

[mihomes]

Noticed this the other day... for the longest time my robots.txt followed the format of :

User-agent: *
Disallow: /test/
Allow: /
Sitemap: [test.com...]

which is consistent with what Google shows you in their webmaster tools excluding the sitemap field.

Now, on robotstxt.org I noticed they state that there is no 'allow' field and the only requirement is atleast one 'disallow'. So in other words the above example would be changed to :

User-agent: *
Disallow: /test/
Sitemap: [test.com...]

What is everyones thoughts on this? In theory it sounds like they accomplish the same exact thing, however, is the 'Allow: /' necessary... or on the flipside... would it hurt to have it?

Would like to hear opinions on this.

[g1smd]

Some searchengine bots understand "Allow". Most bots do not.

I believe you need a blank line before the sitmap line.

Use example.com in this forum to suppress the auto-linking function. See RFC 2606.

[mihomes]

"Some searchengine bots understand "Allow". Most bots do not."

- Yes, I realize that. That wasn't really what I was asking.

You DO NOT need a line before the 'Sitemap:' for anyone reading this.

[lucy24]

which is consistent with what Google shows you in their webmaster tools excluding the sitemap field.

It must be buried in WMT somewhere, because I'm pretty sure that's where I learned you can put the sitemap info in your robots.txt. (Look! I have a sitemap in the exact location and with the exact name you would have looked for if I hadn't said anything!)

I was going to find the specific place and quote them, but g### flew into a rage because I wouldn't give them my cell phone number-- an easy decision to make, since I don't have a cell phone-- and now they are treating me to the perpetual-motion spinning thingie. I can sign out, but if I sign in they just pick up where they left off.

:: sigh ::

On second thought, maybe it was bing.

:: search, search ::

Nope, can't find it there. (But am very intrigued by the notation "URLs listed below are currently blocked by your site’s robots.txt. URLs marked with ! are considered by Bing to be important or significant for your site. Please ensure these are not blocked unintentionally." They put this up even if you don't actually have anything marked with a !)

Awright, g###, time's up. Ten minutes is not a computer glitch, it's the sulks.