Forum Moderators: open
Does Freshbot ever expire? I have no idea what could have caused that page to completely disappear from the index.
I used to be able to search for a unique phrase on that page and google who show that page (we also used to score #6 for the keyword). Now, no such luck - it's completely gone. And I was hoping that the page would have improved PLUS our subpages (which have lots of unique content on them as well) would have been indexed.
Help?
I was under the impression that about the only way to get dropped from google is a major, hand applied, spam penalty - and we haven't done anything like that. I know freshbot results can jump up and down - but completely disappear?
I haven't heard of the site being down at all over the past few weeks...
Thanks,
Chris
On the other hand it seems that freshbot in the last couple of months has been doing a bit of deepcrawlers work. Pages found by freshbot after the deepcrawl may sometimes be indexed without being visited by deepcrawler. I have seen that for a few pages. Other webmasters have reported it too.