| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Google maximum characters robots.txt ?|
Does Google only accept a robots.txt of a certain maximum length?
I have just looked through my robots.txt as displayed by Google webmaster tools 'Crawler access' and it is cut off approximately 40% through.
My robots.txt file is just 13kb
This appears to be a bug whereby the validator tool accepts the whole robots.txt contents except when secondary bots are selected from the bottom drop down menu.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved