Welcome to WebmasterWorld Guest from 54.226.238.178

Forum Moderators: goodroi

Message Too Old, No Replies

Google maximum characters robots.txt ?

     
10:32 pm on Nov 19, 2009 (gmt 0)

Full Member

10+ Year Member

joined:May 24, 2005
posts: 211
votes: 0


Does Google only accept a robots.txt of a certain maximum length?

I have just looked through my robots.txt as displayed by Google webmaster tools 'Crawler access' and it is cut off approximately 40% through.

My robots.txt file is just 13kb

Many thanks

4:54 pm on Nov 20, 2009 (gmt 0)

Full Member

10+ Year Member

joined:May 24, 2005
posts: 211
votes: 0


This appears to be a bug whereby the validator tool accepts the whole robots.txt contents except when secondary bots are selected from the bottom drop down menu.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members