Welcome to WebmasterWorld Guest from 54.226.241.8

Forum Moderators: goodroi

Message Too Old, No Replies

Google maximum characters robots.txt ?

     

cyberdyne

10:32 pm on Nov 19, 2009 (gmt 0)

10+ Year Member



Does Google only accept a robots.txt of a certain maximum length?

I have just looked through my robots.txt as displayed by Google webmaster tools 'Crawler access' and it is cut off approximately 40% through.

My robots.txt file is just 13kb

Many thanks

cyberdyne

4:54 pm on Nov 20, 2009 (gmt 0)

10+ Year Member



This appears to be a bug whereby the validator tool accepts the whole robots.txt contents except when secondary bots are selected from the bottom drop down menu.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month