Multiple URL forwards to primary domain?

multiple URL forward redirect

[notbound]

Scenario:

Have over 30 URL's registered that forward to "primary".com domain. So, example, if you type cows.com in your browser, it autoforwards to the main site farmer.com.

On the farmer.com site, I've got over 40K static pages, that are accessible by a index, each page has links back to the 30 URL's with the specific page as the link.. Example: on the milk.htm page, it has links: farmer.com/index/milk.htm, cow.com/index/milk.htm, etc. These all go to the same milk page on farmer.com... Got the idea?

OK, now I've submitted all 30 URL's in google.com... The farmer.com site is currently being crawled at over 4,000 sessions per hour. It's been steadly doing this for 5 days. My guess is each URL is being spidered.

I wonder if all 30 URL's will show up in google? If you search on milk, you'll get farmer.com/index/milk.htm and cow.com/index/milk.htm, etc.,etc.

Has anyone ever tried this? At the current crawl rate, it'll take over 40 days to finish the crawl?

[Yidaki]

>At the current crawl rate, it'll take over 40 days to finish the crawl?

Could be that you'll get booted before google finishs crawling ...

[cjtripnewton]

Sending spiders in circles is not a good idea.

[kevinpate]

...wondering why anyone would set up 30 links that would redirect a site visitor right back to the page he or she starts on.

If I were at your milk.htm, saw a link for more milk info, clicked and found myself right back at milk.htm, gave benefit of doubt and clicked a second link, only to again feel like Dorothy mumbling there's no place like milk.htm, there's no place like milk.htm ... I'd never try any of the other links and your site would go into the not worth another look file. Potential SERP positioning improvement at the expense of wasting the time of your customers after they arrive seems an odd business priority, but maybe that's just me.

If I want to run around in circles chasing my own tail, I'll decide to believe in reincarnation and come back as a puppy, not roll around in circles on someone's site.

I don't know how the spiders would react, but I can't believe I'd the only consumer who would feel angry about clicking a link for more info only to find myself redirected right back at square 1.

[notbound]

Why not good idea for spider circle? The links back to the same page is for the search engine, not the customer. The links are just "advertisements" for the customer, they would never click on them. It's just for the spider to show each individual page is a?qualified? URL page, even though it redirects to the primary domain. This is the first time I've tried something like this. Basically, we should get 30 URL's for the same content, but different domain names in the engine? Have tried this once with only two URLS, worked... showed www.farmer.com/index/milk.htm and www.cow.com/index/milk.htm in the search results, taken up spots 1 and 2.

BTW, looks like the sessions have started to slow down from over 4K per hour to 2700 per hour. When I just checked, it was averaging 188 sessions per 5 seconds, but then spiked back up to 220...

NOTE: Domain name's referenced are for example only, they are not the real ones...

[Yidaki]

>Domain name's referenced are for example only, they are not the real ones...

LOL

<added>

>Basically, we should get 30 URL's for the same content,
>but different domain names in the engine

yeah, that's a better way to get booted. your actual way may take a bit longer allthough it's strong enough to bring you out of google after a while.

</added>

[jpavery]

who has a 40,000 page site? I guess you do... do you work for Britannica or...? poor little me with my 200 page site...
sitcky me the URL - I would love to see it....
JP

[Huck007]

notbound,
interesting, although very strange way of getting multiple domains listed with the search engines and all pointing back to your site.

Can anyone tell us why this would be a bad idea? ...or at least why it would get you booted from Google? My thoughts are that it would not be a wise thing to do - but I do not have the experience or examples to prove it...but I am sure one of you out there has a good answer.

[yetanotheruser]

notbound,

how are you doing the re-direct? surely google know's about most re-directs and will just end up listing the one domain?

IMHO it's going to either going to get you into trouble with the bot... or it'll just end up indexing one domain if it sees the others are effectively duplicate content.

(A word of warning though.. my previous employer who manages a 'k or so sites tried to be clever with re-directing pages to deal with different language versions... Was a legitimate thing a teckie did to save a lot of hassle, but GoogleBot saw it as cheating and 6/700 well placed sites dissapeared in one quick turn on the dancefloor.. .. seems a little dangerous to me.. )

(you could sticky me the url too :) .. 'cos I'm still not sure I quite get what you're doing..)

ATB, :)

[Yidaki]

>but I do not have the experience or examples to prove

Would you like to run a search engine that returns all the same pages for specific keywords? Duplikate content is annoing and lowers the search quality, is unfair and creates a bad user experience.

[notbound]

how are you doing the re-direct?
Domain is configured to forward, through the registar. No scripting used to forward, done automatically by the registery service. I knew about the java redirect issues, but I haven't seen people doing this way, yet. BTW the site is still being crawled after 5 days at an average of 2400 sessions per hour, all by googlebot.

The 30 URL's are configured at the registery service to autoforward to the primary domain, when someone type's that address in the search bar of the browser. Example: type cow.com, it forwards to farmer.com - all done by the registery service, not scripting.

[yetanotheruser]

all done by the registery service, not scripting.

Then I would hazard a guess that if it's a DNS redirect, which is I think what you're describing, google will deal with this, and shed the other domains? .. if it's the server doing the re-direct.. same goes IMHO..

(If it's not a silly question.. .. why are you redirecting the same page to various different domains?)

[NickCoons]

notbound,

It sounds like what you're doing is for increasing your rankings only, and not for the user browsing your site. Google will catch on to this, and all but one (or all) of your domains will likely be dropped from the index for providing duplicate content.

[notbound]

Yes, to increase search engine ranking based on the keywords within the URL... I've successfully done this approach with 5 different URL's, "DNS" forwarding to the primary and stayed in google for over 5 months, now. Submit all 5 URLS every other week to google, and they stay. xx1.com, xx2.com, xx3.com, etc all show up in google, when you click on them, they goto the primary xx.com. The primary one page site, has the xx1.com, xx2.com, etc., as links in the page back to the home page. Did this approach with only a one page site. Now doing with a 40K page site... Still at avg 562 sessions per 5 seconds by googlebot

[reuben101]

Sounds like you caught yourself a spider...
Almost sounds like you did it on purpose for some odd reason.

I often wonder why folks go to the time and expense to set up elaborate scenarios like this. From a usability standpoint they are horrible and I can't imagine visitors being appreciative of the hassles. I suppose there may be some short term gain to be had, but you must know the seach engines and directories hate that stuff and a major goal is to find sites like yours and remove them. Why not use your time and talents to create something useful and lasting?

[notbound]

Looks like the sessions came to a stop. No more crawl.. Well, now wait and see what happens, if anything at all.

[EquityMind]

I would love to see the outcome of this. I will keep my eye on this...can you sticky me the URL also?

[notbound]

Hahaha... Sorry, guys no URL... Gotta protect it, ya know? But, I think if this does work, which I'm pretty much guessing it won't for a long time hall, you could bet I'm gonna package it up! :[smilestopper]) Even if it only works for a short-time, google links get gobbled up by others, ya know? I'll keep monitoring and keep yall posted.

For the newcomers: Submitted 30 different keyworded URLS that "DNS" forward to a primary domain to google. For the past 5 days, averaged 2K sessions by google bot on a 40K static page primary site. Each individual page of the 40K, link back to each of the 30 different URL's. Each "static" page has the keywords of the URL links in metatags, page content, etc - optimized for the search engines... Have tested same scenario with 2 URLS pointing to one site, and have maintained google ranking for 5 months. Now goes the test on 30 URLS and 40K page site.

[notbound]

ITS BAAACCCKKK! Googlebot came back and is at 1,370 sessions/hour. Must be one of the 30 URL's submitted. Geeze, this could be a infinite loop! Maybe a downfall of submitting 30 URL's that DNS forward to the primary?

[kevinpate]

An itsy bitsy spider
crawled through a round link pop
landed back at square one
then found it couldn't stop
tried another link and
just went round again
That itsy bitsy spider
may soon not be your friend

[notbound]

This morning at 7:27AM CST, reports show 1,290 sessions/hour, all still googlebot. The bot has been steady crawling this site now for 5 days, from different DNS directed URLS. Anyone done anything like this before?

[jaspen meyer]

not sure

[yetanotheruser]

I'm still not sure I see where this gets you? .. I'd be very surprised if google doesn't notice that it's being re-directed..

If it doesn't then surely it's going to notice 30 x 40,000 duplicated pages? If the 30 duplications were something harmless like copies of the Apache documentation I'm sure it would just ignore it and give you the benefit of the doubt... but given that all 30 copies are on the same IP I would think they'll just stop GoogleBot going to your server at all..

I don't know.. doesn't seem quite fair to poor GB to me ;) sending him round in circles like that.. .. besides.. if it works your going to end up with 30 identical links in the search results.. someone will be sure to point this out too G.?

Oh well.. do let us all know how it goes next dance..

ATB ..

[notbound]

Hmmm.. Same IP, duplicate page contents, different URL, same IP? You would think GB, would pick this up immediately. Then why do you think, I'm still getting crawled by GB at 3568 sessions/hour, as of right now. Why do you think I've got a site currently on GB for the past 5 months, with same IP, but 5 different URLS? I'm watching the stats as we speak (gotta neat script that meta refreshes the session hits, so I can realtime watch) Yall are right, though, about making surfers mad,when they see 30 different URLS, but they all go to the same page in the search results. But, 9 times out of 10, with the kind of products I'm selling, the first page hit will be where they buy. There is no price comparison in my market, just whomever gets the hit. So, by taking up all links on first page results, I'm almost guaranteed to "steal" the market. Yea, I know this is bad... But, if I'm not doing it, somebody else will! Why shouldn't it be me - that beats my competition to tha punch? Anyway, GB still crawling, very anxious to see how this turns out.

[notbound]

9:59 PM CST, only 3 open sessions, maybe GB has stopped crawling? Now the patience is needed?

[notbound]

SAMPLE OF LOG FILE: See the times?

538759438 3:44:20 PM 216.239.46.167
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/cow.htm

538759439 3:44:23 PM 216.239.46.193
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/cow2.htm

538759441 3:44:28 PM 216.239.46.23
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/ cow3.htm

538759442 3:44:28 PM 216.239.46.23
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/ cow4.htm

538759440 3:44:28 PM 216.239.46.23
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/ cow5.htm

538759443 3:44:28 PM 216.239.46.23
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/ cow6.htm

538759444 3:44:29 PM 216.239.46.193
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/ cow7.htm

538759445 3:44:30 PM 216.239.46.193
Referer: Direct Request
Browser: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Entry Page: /farmer/ cow8.htm

[notbound]

GB's back, tell a friend.. Guess who's back, guess who's back... I've created a monster, cuz nobody wants to see google crawler no more they want see my URL, I'm chopped liver. Well, if you want my URL, then this is what I'll give ya, a little bit of milk mixed with some hard chicken eggs... :) Oh, well for my singing... Google's crawling the site again at 2/14/2003 8:53:55 AM , 2,764 sessions/hour. I'm not sure if it will ever stop. Has ANYONE tried this scenario before? Should I stop the crawler somehow?

[notbound]

Over 10 hours, no GB... Anyone guess what's going to happen next? Do you think all 30 URL's will get index? I'm anxious to see what will happen...

[fathom]

So are we! :)

[notbound]

12 days, still not ANY of the 40K pages are showing up in google. Freshbot has come back, hitting various pages out of the 40K, but still not any of them showing up. Recap: New site - 40K static pages - all crawled by google, in the last deep crawl - fresh bot has come back - nothing showing up?

Any ideas why?