Forum Moderators: open

Message Too Old, No Replies

deleted old pages but spiders see it as errors as it's no longer live

         

hulahoop

11:50 pm on Jan 18, 2006 (gmt 0)

10+ Year Member



Hi,

I have deleted some old pages from my site that not relevant. The spiders don't see it there any more and report an error in my google sitemap. I don't want to go to google to submit a page removal.

My question is, does it do any harm to your site in terms of PR and google's view of you if you have some 404s found or HTTP errors.

Or does it just take those pages out eventually? TQ

Dijkgraaf

4:43 am on Jan 20, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've had spiders request pages that have been deleted for years afterwards.
The best practice I have found is to initially redirect the old URL to the new one (if there is one), and after a few months add the old URL's to robots.txt file so that they aren't requested again after that.

hulahoop

4:39 pm on Jan 20, 2006 (gmt 0)

10+ Year Member



I have only deleted pages I don't want anymore. So does that mean I should add them in my robot.txt? I assume aftersome time, this will prbably grow quite long right?

Still trying to find what is the best way to do it.

So does a 404 have any effect on Search Engines in terms of your placement and ranking?

incrediBILL

8:05 pm on Jan 20, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Why would your Google sitemap return an error unless those pages are still listed in that site map?

D'OH!

hulahoop

12:25 am on Jan 21, 2006 (gmt 0)

10+ Year Member



Google apparently uses the sitemap as a guide but spiders whatever they find still.

pocpocpoc

7:13 am on Feb 9, 2006 (gmt 0)

10+ Year Member



You should be returning a 410 "Gone" response for pages that are permanently removed.

[w3.org...]
"410 Gone. The requested resource is no longer available at the server and no forwarding address is known."