| 9:21 am on Aug 21, 2004 (gmt 0)|
Sitematch will get it recrawled quickly if the change the content that would work. However you could also ask them to mod_rewrite the URL to somewhere else or block yahoo in their robots.txt for that particular page
| 7:53 pm on Aug 21, 2004 (gmt 0)|
|...However you could also ask them to mod_rewrite the URL to somewhere else or block yahoo in their robots.txt for that particular page |
The issue isn't blocking Yahoo from spidering existing content. All the problematic content is on a single page, and that page can be entirely removed or changed.
The issue is updating Yahoo's index and getting the old data overwritten as quickly as possible. What's faster...
- dropping the page and free submitting to get a 404...
- changing the content but keeping the page and submitting the page to Site Match?
Can you just submit one page to Site Match? Its only function would be to overwrite the old data. Would they let you submit a page that just said "page content deleted"?
| 7:14 am on Aug 22, 2004 (gmt 0)|
The free add url is another option and will probably have a similar effect and may save you a few dollars. If that doesnt work Site Match may be a good idea.
| 9:22 am on Aug 22, 2004 (gmt 0)|
Pages that are deleted should be served with a 410 Gone status, not with a 404.
My experience is Google understands a 410 really fast, Yahoo seems to need a few weeks to remove the page from their index.
| 8:09 pm on Aug 22, 2004 (gmt 0)|
Tim - Thanks. With either free add url or with Site Match, which will be recognized faster - a dropped page or a changed page?
Span - Thank you as well. It's not my site, and I would bet that the organization whose site it is doesn't have a webmaster capable of setting up a 410, but I'm filing the thought away for reference. Just found this recent thread on the subject of 404/410s, possibly of interest...
Should I use 404 or 410 for GONE for HTTP 1.0 search spiders?