Forum Moderators: open
Since I made a little error in my meta tags, namely <meta name=robots conteXt=index,follow> instead of <meta name=robots conteNt=index,follow>, Googlebot hasn't come back to index my pages, which it used to do almost every day in the past.
Some of my clients' websites which had the same meta tag error, have the same problem, even if the error was corrected a month or so ago.
Strangely, the websites of my clients that did not have this error, still are indexed on a daily basis.
Should I worry here? What can I do to solve this problem?
<>
Greetz,
Alex.
[edited by: ciml at 12:02 pm (utc) on Oct. 20, 2003]
[edit reason] Generalised. [/edit]
> error in my meta tags, namely <meta name=robots conteXt=index,follow>
Since that line does not contain "content=", I suspect Googlebot will ignore it and default to the "index,follow" behaviour, just as if the tag was not there.
The difference in the crawling behaviour on this site and the others you mention may be due to other factors, such as number of incoming links from pages that Google knows about, the PageRank of those pages linking to your pages, and the resulting PageRank of the pages of your site. If your site is PR3 or below, then it probably won't get crawled very often - most likely once a month. If your site is PR4 or above, it will probably be crawled more often.
I'm using "probably" in the statements above, because Google changed a lot this year, and I'm not comfortable yet saying I'm certain about its new behaviour patterns.
If Googlebot *did* do "the worst thing", and assume you did not want the site crawled, it will almost surely try again within 30 days. I would advise you to use the time to develop more incoming links with on-topic link text from high-PageRank pages of sites related to your market segment, and work on your on-page factors to improve your ranking in the search results. Either you don't have a problem, or you have a problem that will fix itself. Either way, use the time to improve your long-term results.
Jim
