Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: goodroi
How can I get Google and other spiders to only index our main site? While we appreciate our partners, these aren't really valid referrals. We'd obviously love to stay ranked well for those words.
[edited by: rogerd at 5:07 am (utc) on Dec. 23, 2004]
[edit reason] No specifics, please. [/edit]
The consequences may not be all positive. First, you'll lose showing up several times in the search results. Second, your affiliates won't show up at all, ever, in search engine results. This may not please them. They may even need to set up independent websites to cope. Then you'll be back where you started, with less control.
If you want to use robots.txt, a good guide is:
I don't want to block the spiders completely though from any files. I guess what I'm looking for is a way to tell the spiders, "only index files on www.billingorchard.com". Is that possible?
If I did something like check for the referring URL and then redirect to our main site if it's coming from Google, would that inhibit Googlebot from spidering our site? When spidering, I'd assume there's no referral from Google, correct?
Thanks again for your help.
Your original pages will be (are) indexed and you should rank where you want with no one else pointing to them. Many manufacturers have completly different domains for reseller content for this very reason.