Forum Moderators: open
I’ve just been cycling through my server logs and found spiders calling pages that do not exist on my server.
An example of what I would like to stop:
Search engine robots are requesting files in this structure:
Mydomain.com/index.htm/directory1/directory2/
Now from my limited experience I understand that the server is treating this as a query. The “/” after my index.htm is becoming a parameter.
I’ve checked and we never use the “/” as a parameter on our server so is there a way I can remove this Character so my server will serve my default error page.
I’d like this to happen if possible:
Mydomain.com/index.htm/directory1/directory2/ to return a error page
And
Mydomain.com/index.htm?directory1/directory2/ to return a ok response
Is it as simple as editing a file and removing the “/” from it, if so where would that file be located.
Any help on this would be greatly appreciated
Vimes.
Now in IIS Mgr -> Website Properties -> Custom Errors Tab -> Highlight the HTTP Error for 404 -> Edit Properties -> Message Type=URL -> URL= [yoursite...]
All requests that now do not return a valid file/path name will see the 404default.htm page.
One good idea is to make your 404 page return a Header of '404 Page Not Found' so robots that will honor a 404 and stop looking for it will get a status=404 rather than 200.