Forum Moderators: open
Now over a month ago I inadvertently set the robots tag to 'none' on a whole section of a dynamic part of a site, I since corrected this but still google is not indexing it (wheras other robotos, such as FAST are), whilst it is still regularily updating the dyanamic part that was not affected.
My question is thus - does google cache the robots tag/file and if it finds a negative instruction then increase the time that the robot will next visit that page to see if the tag has changed and reindex (if ever...)?
Cheers,
// Jacob