| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|Google maximum characters robots.txt ?|
| 10:32 pm on Nov 19, 2009 (gmt 0)|
Does Google only accept a robots.txt of a certain maximum length?
I have just looked through my robots.txt as displayed by Google webmaster tools 'Crawler access' and it is cut off approximately 40% through.
My robots.txt file is just 13kb
| 4:54 pm on Nov 20, 2009 (gmt 0)|
This appears to be a bug whereby the validator tool accepts the whole robots.txt contents except when secondary bots are selected from the bottom drop down menu.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved