Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Google maximum characters robots.txt ?



10:32 pm on Nov 19, 2009 (gmt 0)

10+ Year Member

Does Google only accept a robots.txt of a certain maximum length?

I have just looked through my robots.txt as displayed by Google webmaster tools 'Crawler access' and it is cut off approximately 40% through.

My robots.txt file is just 13kb

Many thanks


4:54 pm on Nov 20, 2009 (gmt 0)

10+ Year Member

This appears to be a bug whereby the validator tool accepts the whole robots.txt contents except when secondary bots are selected from the bottom drop down menu.

Featured Threads

Hot Threads This Week

Hot Threads This Month