Welcome to WebmasterWorld Guest from 54.144.79.200

Forum Moderators: phranque

Message Too Old, No Replies

Domain 404 Page Issue

     

SunnyG

6:42 am on Sep 23, 2013 (gmt 0)

5+ Year Member



Hello members,
Hope all of you are doing fine.

What's your best way to solve 404 Error Pages of your website ?

By mentioning pages in Robots ? Or there's another way ?


Thanks for your help in advance.
SunnyG

phranque

7:03 am on Sep 23, 2013 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



what is the problem you are trying to solve?
if a requested url doesn't exist it is appropriate to return a 404 response (or 410 in some case) so not all 404s are a problem.

SunnyG

7:23 am on Sep 23, 2013 (gmt 0)

5+ Year Member



phranque thanks for your prompt answer :)

Actually a week back I was reading this somewhere that we should mention 404 pages in robots and block the SEs. So I got confused and raised the question here.

Btw if there are so many pages then what we should do ? Most of the times We see it in Google Webmaster Tools for some sites with loads of pages.

Thanks
SunnyG

lucy24

9:07 am on Sep 23, 2013 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Are you asking about the 404 page itself? Sure, slap a "noindex" meta on it if you're worried. But I don't think search engines index error documents; they probably don't even read them.

:: detour here as I realize I've got a very easy way to check ::

I have a stylesheet that's used only by my error documents; almost nobody sees it except humans who got locked out by mistake. afaik the googlebot has never asked for it-- that's what I just detoured to look up-- though they routinely get my other stylesheets. That can only be because they don't even look at the 404/410 page, but just note the header. (The plainclothes bingbot constantly asks for the file, but that's just part of the act.)

Don't block error documents in robots.txt; that could be counterproductive. Besides, robots don't normally ask for your 404 page unless there's been a mistake of some kind. They get handed it whether they want it or not.

SunnyG

9:20 am on Sep 24, 2013 (gmt 0)

5+ Year Member



Lucy24...can you pls. explain a little more about your stylesheet. Hope you have no problem :)

Thanks

lucy24

7:25 pm on Sep 24, 2013 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



You don't really need to know about the stylesheet. I just mentioned it as something I use for information.

All my error documents-- 403, 404, 410-- call on a single stylesheet that isn't used by anything else. Humans landing on an error page will automatically get the css-- and also the favicon or apple-touch-icon, if they haven't already got it. Robots don't normally get stylesheets. But the major search engines do, because they index them along with everything else. So my thinking is that if google has ever physically seen any of my custom error pages, they would have learned that the css exists and they would eventually ask for it.

When your server sends out a 4xx response, it automatically sends out the appropriate error document. But a robot can choose not to look at the document.

If you don't use a separate stylesheet for error documents, and you don't have a "noindex" header, you could try an exact-text search for something from your error document and see if it turns up. This will of course only work if there's something unusual about the wording.

But we still haven't nailed down the original question. Are you asking how to keep search engines from indexing your 404 page? Or are you asking something entirely different?

SunnyG

11:24 am on Sep 25, 2013 (gmt 0)

5+ Year Member



Ok. Actually I wanted to if your site got lots of 404 products Pages which is no longer required then what do you do to lessen the numbers.

Thanks

jamesMP

11:36 am on Sep 25, 2013 (gmt 0)



404s aren't necessarily a bad thing - they tell people (or search engine bots) that the page they're looking for doesn't exist at that location.

If the page genuinely doesn't exist, then its ok (and best practice) to continue to serve a 404, however if the page *does* exist and you've changed its url, then you should set up a 301 redirect from the old url to the new one.

phranque

5:17 pm on Sep 25, 2013 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



if a previously valid url is no longer being served it is appropriate to provide a 410 Gone response.
these 410s will eventually drop from the GWT list.

lucy24

9:01 pm on Sep 25, 2013 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



o/t but I have to add this since I brought up the subject earlier:

After saying categorically that the googlebot has NEVER asked for my error-document stylesheet-- implying that they don't know it exists-- only yesterday I found them asking for it. (Technically Monday, but I discovered it yesterday.) Coincidence, sure, but a slightly creepy one.

SunnyG

7:22 am on Sep 26, 2013 (gmt 0)

5+ Year Member



Thanks phranque , I think that's what I wanted to know. Thanks to all for their time to answer.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month