tantalus - 6:13 pm on Apr 1, 2010 (gmt 0)

Just throwing this out there but I had this problem with msn and yahoo a couple years ago when they changed the way they handled robots txt. All my traffic completely vanished and I thought I had been banned at the time, it was only when yahoo went the same way I figured it out. Google was always fine with the robots txt that I had. Is Google following suit?

From a quick look what I am is seeing is no "allow directive" ie...

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/

Instead explicitly allowing as in ...

User-agent: Googlebot
Allow: /
Disallow: /cgi-bin/
Disallow: /shop/
Disallow: /dev/
Disallow: /admin/
Disallow: /private/


Just a suggestion.