HELP!--My search engine results are a mess!

[ichthyous]

Hi there, I run my own business and designed and manage the web site as well. Occasionally i go into the apache config files but I am by no means a programmer. My problem is this: I have 6 domain names registered, 5 of which all just forward to the main one. The search engines keep crawling the 5 forwarded domains and come up with strange results that i don't want them to show, for example:

I want the SEs to crawl www.abc.com and show pages a.html through x.html or www.abc.com/athroughx.html BUT what is really happening is that they are crawling www.def.com, ghi.com, jkl.com, etc. which forward to my site and are producing the following search engine results:

www.def.com/athroughx.html
www.ghi.com/athroughx.html
www.jkl.com/athroughx.html

When a viewer tries to click theses results they get a "page not found" error. I need to make sure that the search engines crawl those domain names they produce the right results. A programmer on here gave me some apache aerver code to force the spiders to always only crawl www.abc.com/athroughx.html. I'm not sure it ever worked and then it started to crash the server so i had to remove it. Does anyone have any advice on how to code the httpd file to force the spiders to crawl only certain pages? i will post the code that the first guy gave me below. thanks for any help!

---------------------CODE-----------------------

RewriteCond %{HTTP_HOST} "!^www\.the-right-domain\.com"
RewriteRule (.*) [the-right-domain.com...] [R=301,L]

[jdMorgan]

ichthyous,

Welcome to WebmasterWorld [webmasterworld.com]!

That code is basically correct, although the quotes aren't needed in the RewriteCond line.

RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.the-right-domain\.com
RewriteRule (.*) http://www.the-right-domain.com/$1 [R=301,L]

If the code is properly installed and working, you will see the following: If you request any page from any of your domains except "www.the-right-domain.com" will be redirected to the same page requested, but at "the-right-domain" instead, and you will see your browser's address bar change to show "the-right-domain". If you don't see that, it's not working.

Jim

[ichthyous]

I used to have this code in place in the htacess file and it worked fine. But then the tech guys at my hosting company made "changes" to my server and now it wont run. They got blitzed by the hurricane that hit the east coast and couldn't manage to get my server back up for 10 days and when they did i was now configured as www.example.html not www.example.com...I suspect that this is the problem as the code worked fine until that point. I think their server blew and they have rigged me up in some sort of cheap way that is killing me SE wise. Does anyone have insight into what might be going on or how to rewrite the code to fix it? Thanks again!

[edited by: jdMorgan at 6:34 pm (utc) on Oct. 24, 2003]
[edit reason] Generalized domain name [/edit]

[jdMorgan]

ichthyous,

If your domain is resolving as www.example.html, you have a serious problem, and your hosting service needs to fix it. If they can't or won't fix it, you need a new hosting company.

Your code was simple, essentially correct, and should work on just about any Apache server with mod_rewrite capability. Maybe your DNS is misconfigured -- it's a very strange problem.

Jim