Welcome to WebmasterWorld Guest from 54.145.123.86

Forum Moderators: phranque

Message Too Old, No Replies

Correct HTTP response for domain not found

If the server does not recognise the domain is 404 correct?

     
8:07 am on Nov 25, 2016 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 16, 2005
posts:2660
votes: 85


I have code that runs multiple sites, so it uses the domain to decide what content to serve.

If it does not recognise the domain it returns a 404.

This seems to be correct as the definition is:

The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.


[w3.org...]

It says URI, not path, so it looks correct. On the other hand, it does not feel right to give no indication that you have the domain wrong rather than the path. I cannot see any more appropriate response, short of 418...
8:20 am on Nov 25, 2016 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:9228
votes: 439


Is this about server response or what the user sees? If the later, you could set up a custom 404 page for each domain giving more complete info.
11:38 am on Nov 25, 2016 (gmt 0)

Senior Member from KZ 

WebmasterWorld Senior Member lammert is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 10, 2005
posts: 2933
votes: 20


Besides the technical question how to respond to traffic, you should also question yourself where the traffic is coming from. If it is a significant amount of human traffic which you receive through non existent domains (I assume these are sub domains?) you may try to monetize it as well. If the traffic is mainly bots and you want to send them away permanently then just dropping the packet and not responding at all would be the best solution IMHO. The domain does not exist, has probably never been active so why send HTTP status codes for something which isn't there in the first place? Sending status codes costs your resources and gives the impression to the bots that something is there which will make them try again in the future. I know of no single bot which doesn't re-request an URL after they have been fed a 404 or 410 status code.
7:06 pm on Nov 25, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13902
votes: 491


On the other hand, it does not feel right to give no indication that you have the domain wrong rather than the path. I cannot see any more appropriate response, short of 418...

Who is the visitor? Do you have any indication that requests for the wrong domain are coming from bona fide humans, rather than from malign robots? (If they're from select regions such as Brazil or Russia, even an apparent human is more likely to be an infected browser.) If so, can you pinpoint the referer responsible for sending them to the wrong place? People don't just make up domain names out of a hat; there's got to be a DNS pointing to your server or they wouldn't arrive in the first place.

The ordinary way to handle requests for the wrong domain name is to 301 redirect to the correct name. If for some reason you don't want to do this (essentially you're telling the malign robot what domain “really” lives at their requested IP), then a 404 is generally the best response. It's like returning a manual 404 response to requests for login.php: the robot receives absolutely no information other than that these aren't the droids you're looking for. If, on the other hand, you return a 403 or 418 or 4-almost-anything, the robot may get suspicious and wonder what you're hiding.
4:27 am on Dec 7, 2016 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 16, 2005
posts:2660
votes: 85


I actually have more than one system with these issues. I think they require different responses (because where the traffic is coming from etc. differs)