Forum Moderators: open

Message Too Old, No Replies

Google crawling the wrong server

         

Gibble

2:10 pm on Sep 30, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I have a development server, and a production server for my website, both world accessible but google for whatever reason likes my development server better. The problem I believe is that both sites are virtually identical so when it gets to my produciton site it appears to be duplicated content.

It spidered nearly 500 pages on the dev server and only 5 on my production server. This is NOT good. I have updated the robots.txt files to keep it off the development server (other spiders don't appear to be following its rules though)

I want to make sure the next crawl is done properly on the right server. What should I do? How can I get googlebot to stop spidering my devlopment server and spider my production server instead?

korkus2000

2:32 pm on Sep 30, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Here is a recent discussion about a similiar problem:
[webmasterworld.com...]