Robots Exclusion Confusion

[STMLJohn]

Does anyone know how Google copes with a robots.txt that contradicts itself?

for example...

User-agent: *
Allow: /widgets
Disallow: /

should it really read

Disallow: /bluewidgets
Disallow: /redwidgets

and so on, until all disallowed directories are listed.

I asked as I have noticed a site the Google appears to be ignoring the REP for in the former format. Perhaps the logical contradiction confuses Google. Does anyone know?

Thanks

[Brett_Tabke]

The allow line is thrown out as there is no allow line in the standard. note: other engines may throw the entire site out, or include the entire site. Robots.txt errors are not something you want on your site!

[STMLJohn]

Thanks Brett - that's what I expected. However, the whole site appears to be indexed, which makes me think Google is ignoring the whole REP.

The REP is on a competitors' site, and I wanted to get my head round it to avoid similar pitfalls on any of my sites.