Google Problem

GoogleBot got Error 401

[neh2008]

I have had a very strange incident with googlebot and scooter.
They both came to my site one by one (next to each other) and looked for robots.txt 3 days ago.

They could not access the file and got error code 401 in return......:(( i don't know how, but I fixed my fault. I hope the permissions are set now...At least I can see the file now....I verified it too....

The question is, how does ERROR 401affect GoogleBot and Scooter? What is their Reaction?

Please help.....

[jdMorgan]

neh2008,

My guess... They go away and come back later. No search engine in the world has enough memory to "remember" all the sites they get an error from. :)

Jim

[o0_cops_0o]

I thought you only need a robot.txt in your code to keep away spiders on some pages you dont want access. If you dont want to keep the spiders out of any pages you simply dont add the robot.txt. I find the deep crawl spidering my pages and looking for the robot.txt, gives me an error because it cant find it, than continues to spider my site

[jdMorgan]

cops,

He got a 401-Authorization Required "error", due to misconfigured file permissions.

But just like a 404-Not Found error, I'd expect the 'bots to simply come back and try again later.

Jim

[o0_cops_0o]

ahhh ok. Thanks, I've been on this site for a few days now and i cant believe how much im learning from you guys. Thanks

[neh2008]

If it comes back, as you said..I should be all set.
If not, my ships are sinking, this is my first ever experience with SEO Project. Hope I get thru it.

Thank you for your co-operation.

~N