Forum Moderators: open

Message Too Old, No Replies

Robots tag caching

         

verseguru

11:23 am on Sep 2, 2003 (gmt 0)



This probably isn't common occurance but maybe someone has had a similar experience and knows the outcome...

Now over a month ago I inadvertently set the robots tag to 'none' on a whole section of a dynamic part of a site, I since corrected this but still google is not indexing it (wheras other robotos, such as FAST are), whilst it is still regularily updating the dyanamic part that was not affected.

My question is thus - does google cache the robots tag/file and if it finds a negative instruction then increase the time that the robot will next visit that page to see if the tag has changed and reindex (if ever...)?

Cheers,
// Jacob