|SEO best practice for expired content|
| 12:56 pm on Sep 3, 2007 (gmt 0)|
On the job section of my website, the content (jobs) expires when positions are filled, and the URL that has been indexed doesn't exist anymore.
I am wondering if I should go for a basic 301 redirect, or if there is a better practice as far as search engines are concerned.
From a user point of view, I guess that a page telling them that the job is not available any longer would be good, but again, what are the implications in terms of SEO? (I often see a page that gives a brief message before triggering an automatic redirect to the home page)
thank you for your help, and long life to WW,
| 4:24 pm on Sep 3, 2007 (gmt 0)|
Best practice is to return what it actually is. Though most SEs treat a 404 (page not found) and 410 (page gone) response the same, I'd go with a custom 410 page telling the user that the listing no longer exists. I wouldn't use any redirection as a large number of pages redirected to the same one, such as the home page, can either look spammy to or at least confuse bots. Include at least a link to the home page or, if programatically possible, a link to a similar job category.
| 9:41 pm on Sep 12, 2007 (gmt 0)|
(late) thank you,
| 11:50 am on Sep 14, 2007 (gmt 0)|
a friend just advised to do a 301 to the job result pages of the category where the job was posted (that means 301 to some sort of similar content I guess).
Would that be better than the 410 in your views?
| 2:05 pm on Sep 14, 2007 (gmt 0)|
301 to related category of jobs or list of related job offers on the subject sounds like the best solution. At least, if I was searching for a job in a specific area then i'd be happy if other options were also provided .. But that's from a user-perspective, not from a SEO..Wait, didn't some of the SE's say that you should build sites for users and not for engines? Engines will catch up on what you're doing and adjust accordingly..
Sincerely, and have fun
| 3:48 pm on Sep 14, 2007 (gmt 0)|
The problem with the 301 redirect is that it doesn't tell the 'bot that the page is gone, nor the user that the job is gone; both can get confused.
Best bet is to return the 410 with a bit of nav help for the user. Some folks use a meta refresh from the error page to the home or other page on the site. This carries the additional problem of how different SEs meta refresh redirects -- 301s, 302s, sometimes 301s, sometimes 302s? 'Bots and indexers aren't always the smartest and most consistent creatures in the digital world; do whatever you can not to confuse them.
| 5:22 pm on Sep 14, 2007 (gmt 0)|
since I know you get interested in the subject, here is another one I just spotted on a site. However, this is confusing to SEs.
One of my competitors (with pretty impressive ranking) leaves pages with a 200 (OK) and a message along the lines of "it doesn't exist any longer" and obviously a bunch of links around that...
it sounds to me that the 410 is the most ethic one, though the 200 or the 301 must work well seo wise (at least short term)
| 5:38 pm on Sep 14, 2007 (gmt 0)|
|A friend just advised to do a 301 to the job result pages of the category where the job was posted (that means 301 to some sort of similar content I guess). |
That would work if the content at the upper level category was relevant to what the user is looking for. If there are a number of job titles in one category and one goes inactive, I see no problem with 301'ing that user back to the upper level and letting the user see other opportunities under that job title.
|Would that be better than the 410 in your views? |
Again, it will all depend on where you put the user. Can you still put them one click away from where they need to be? If so, I'd probably suggest the 301 method.
Here, you have a user already who doesn't mind being redirected back to an upper level...
|At least, if I was searching for a job in a specific area then i'd be happy if other options were also provided. But that's from a user-perspective, not from a SEO. |
|The problem with the 301 redirect is that it doesn't tell the 'bot that the page is gone, nor the user that the job is gone; both can get confused. |
I would think this is open for interpretation. In this case, a 301 is telling the bot that the page is gone and to permanently redirect to a page that can fill the void. Personally, I'd rather put the user at a page that is more helpful than a custom 404 or "Page Not Found".
|One of my competitors (with pretty impressive ranking) leaves pages with a 200 (OK) and a message along the lines of "it doesn't exist any longer" and obviously a bunch of links around that. |
Are you sure its returning a 200? For both bots and humans? If so, that would probably fall in the "stub" category and not be of any benefit from an SEO perspective, or from a user perspective if you don't mind me sharing my opinion. ;)
|It sounds to me that the 410 is the most ethic one. |
Wait, wait! Let's keep ethics out of this please. We're discussing the correct protocol to utilize when pages no longer exist. Which status you choose to return will be relevant to where the user should end up if the page does not exist.
| 5:57 pm on Sep 14, 2007 (gmt 0)|
"Are you sure its returning a 200?"
yes... he's got 40 000 records of the same page in Gg. he leaves all dynamic metas on and just replace the content of the page with a "no longer available" message and all his nav and ads around.
Therefore users can still navigate away from the site (not very user friendly though), and he still gets impressions on his ads (evil)
i guess that's one benefit then...
As per 410 vs 301, it's true that 410 seems to be closer to what the page actually is (which is why I say ethic).
Now as a user, I'd rather be taken to a page of relevant and similar results (easy in my case)
What I am thinking of doing now is returning a 301 and send to the category result page.
I'll check the referrer, and if it comes from an expired ad, I'll display a user friendly message to explain.
(I've changed my opinion too many times today and it start spinning)
| 6:28 pm on Sep 14, 2007 (gmt 0)|
|I would think this is open for interpretation. In this case, a 301 is telling the bot that the page is gone and to permanently redirect to a page that can fill the void. |
That's the problem with this biz, much of it is interpretation. But I just looked up to check because I thought the sky must be falling! I never thunk I'd be quoting the W3C to P1R ;-)
|301 Moved Permanently [w3.org] |
The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs.
My interpretation is that using this indicates to the 'bots that the job listing (requested resource) can be found in a new location. It can't. I personally don't like using the 301 because I really don't know -- and don't care to find out -- the implications of redirecting multiple pages with different page titles, descriptions, content, etc., to one other page. We know the 'bots are easily enough confused as it is, I don't want to add to the possibility.
|10.4.11 410 Gone [w3.org] |
The requested resource is no longer available at the server and no forwarding address is known...
The 410 response is primarily intended to assist the task of web maintenance by notifying the recipient that the resource is intentionally unavailable and that the server owners desire that remote links to that resource be removed. Such an event is common for limited-time, promotional services and for resources belonging to individuals no longer working at the server's site.
That fits tomajacar's situation quite nicely. It also allows for a place to tell the user that "Hey, that job isn't listed anymore, here's some links that might help." My preference is for that approach; I'd rather not have users wondering where they wound up, still expecting to find what they clicked on.
Of course, the next two lines bring up that whole "interpretation" thing again:
|It is not necessary to mark all permanently unavailable resources as "gone" or to keep the mark for any length of time -- that is left to the discretion of the server owner. |
Which, I suppose, can be read as enough justification to 301 'em if you want.
| 7:12 pm on Sep 14, 2007 (gmt 0)|
jimbeetle, you're more than welcome to put me on the carpet at any time. ;)
|My interpretation is that using this indicates to the 'bots that the job listing (requested resource) can be found in a new location. It can't. |
No, not that particular job. But, another job with the same job title in the same geographic area might be. That's where the interpretation comes in.
|I personally don't like using the 301 because I really don't know -- and don't care to find out -- the implications of redirecting multiple pages with different page titles, descriptions, content, etc., to one other page. |
I know what you mean. Let's say you have 50 jobs with all the same job title (it is a regular occurrence). One of those jobs expires. It was up for 90 days, probably enough time to garner some exposure. I might consider redirecting that exposure to other "same job titles" so the user doesn't have to wade through "you might be interested in...".
In a perfect world, all of this would be handled through a variety of technology and put the user no more than one click away from where they need to be. If I had the technical expertise available to me, I'd serve the 410 to the bot but made sure the user was 301'd to a more appropriate page based on their request.
| 5:01 pm on Sep 15, 2007 (gmt 0)|
I've been thinking about this a bit more and came up with something that might be better all around.
Instead of deleting the job, keep it around, at least for a time.
1) When a job expires, insert a robots noindex meta in the head.
2) Include a "This job has expired," notice on the job page.
3) Also include an "Here are other jobs you might be interested in" link.
'Bots get told not to index the page, users see what they wanted and are given an explanation. No confusion on either side.
4) Run a routine that purges the job data to an archive for any expired pages more than however many months old.
5) Now start serving a 410 for the deleted pages.
This gives time for the SEs to drop the pages out of their indexes. The purge might have to be done at 12 months or so since Google only very grudgingly and slowly drops pages once it's found 'em.
Hmmm, I just had a look at jobsearch.monster.com in Yahoo site explorer. From what I can see it doesn't have any individual job pages indexed, only category and location pages. If you can afford to (don't know how much organic SE traffic your individual job pages draw), a robots no index meta on each of theose pages might be a consideration.
| 8:02 pm on Sep 15, 2007 (gmt 0)|
Unfortunately, I am not monster (yet)
With a pretty new site, traffic to job ad pages is key to my traffic...
In regards to a noindex after the job expires, I'd be wary of confusing bots actually.
I regard noindex as a way of telling engines that content is private and should not be shown to the public.
I had a look at craigslist and it seems that they use a 404 and a link back to the HP.
the more I think about it, the more I am leaning toward the 410. It just seems to be the closest to what I am trying to say about these pages.