Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Should I add dead page supplementals to robots.txt

removed via url console

         

suzie250

5:32 pm on Feb 23, 2006 (gmt 0)

10+ Year Member



I've attempted to get googlebot to recrawl pages that no longer exist and in the supplemental index, in which I had a redirect in place. I've had no success with that so I am removing them via the urlconsole.

I suppose since these pages are supplemental, they will come back at some point so.....

After the removal is successful, I thought about recreating blank pages using no archive and no index tags and adding them to the robots.txt file. If they show up again, I could easily resubmit the robots.txt file and get them removed all in one swoop.

Does anyone have other suggestions for keeping them from haunting me in the future?

Thanks,
Suzie

tedster

12:41 am on Feb 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I suppose since these pages are supplemental, they will come back at some point

Not if the urls don't resolve. Those urls may hang out indefinitely as Supplemental before Google finally lets them go, but they should not "come back" if they continue not to resolve.

annej

2:13 am on Feb 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've tried noindex and several other things. It seems once supplimental always supplimental. I hear Big Daddy is supposed to solve this problem. I hope it really does.

bumpski

12:27 pm on Feb 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



From what I've seen it appears the transition to big daddy makes almost all website pages temporarily go Supplimental, then once in the Big Daddy data base they are restored (if worthy).

annej

4:05 pm on Feb 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I hope they restore only the worthy one and drop the web pages that aren't online anymore. If I take a page off of my website Google should be able to figure out it's gone, not there, no longer on the internet. How hard would that be?