Forum Moderators: open

Message Too Old, No Replies

Google Problem

GoogleBot got Error 401

         

neh2008

9:22 pm on Feb 16, 2003 (gmt 0)

10+ Year Member



I have had a very strange incident with googlebot and scooter.
They both came to my site one by one (next to each other) and looked for robots.txt 3 days ago.

They could not access the file and got error code 401 in return......:(( i don't know how, but I fixed my fault. I hope the permissions are set now...At least I can see the file now....I verified it too....

The question is, how does ERROR 401affect GoogleBot and Scooter? What is their Reaction?

Please help.....

jdMorgan

9:30 pm on Feb 16, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



neh2008,

My guess... They go away and come back later. No search engine in the world has enough memory to "remember" all the sites they get an error from. :)

Jim

o0_cops_0o

9:36 pm on Feb 16, 2003 (gmt 0)



I thought you only need a robot.txt in your code to keep away spiders on some pages you dont want access. If you dont want to keep the spiders out of any pages you simply dont add the robot.txt. I find the deep crawl spidering my pages and looking for the robot.txt, gives me an error because it cant find it, than continues to spider my site

jdMorgan

9:40 pm on Feb 16, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



cops,

He got a 401-Authorization Required "error", due to misconfigured file permissions.

But just like a 404-Not Found error, I'd expect the 'bots to simply come back and try again later.

Jim

o0_cops_0o

9:45 pm on Feb 16, 2003 (gmt 0)



ahhh ok. Thanks, I've been on this site for a few days now and i cant believe how much im learning from you guys. Thanks

neh2008

10:39 pm on Feb 16, 2003 (gmt 0)

10+ Year Member



If it comes back, as you said..I should be all set.
If not, my ships are sinking, this is my first ever experience with SEO Project. Hope I get thru it.

Thank you for your co-operation.

~N