How did Yahoo index site w/out IBLs and w/out submission?

[ColdFusionJunkie]

We have an entire domain blocked and is only accessible via internal dns so we could develope and test before publishing to the production server. A few months ago we opened our development server/domain up so one of our developers could work on it remotely and it was open for about 2 months. We don't have any inbound links, nor have we submitted this domain to Y!, yet we are now showing up with ~200 pages indexed and cached.

Do they read the whois to find domains, then constantly ping them for content? We do not have any machines with the yahoo toolbar, or anyone with yahoo accounts (of any kind). The domain is several years old and until a few months ago has never been public. Any ideas?

Another oddity is that the product domain has been banned from the yahoo index for about a year now. We believe it is because of a change from directory structure to subdomains without proper 301's. This has been fixed, but still not indexing.

Should we open the development domain back up and conditionally 301 redirect from the development to production (if internal go to development, otherwise go to production)?

Any thoughts are appreciated.

[trinorthlighting]

Umm, there might have been a link out there via email. I am also sure yahoo crawls whois as well. I would not complain about being indexed, when you go live with the site it will be a big plus.

When every I make a new site (in test mode, the first thing I do after I build the home page is to submit to the search engines.) That way by the time I am finished it is already indexed. Also, yahoo just changed their algo, so that is why the ban might be lifted. If I were you, I would start submitting to google and msn even though you are in a development phase. Make some sort of way for customers to contact you and set up a catch all email address.

Getting indexed is 1/2 the battle....

[ColdFusionJunkie]

trinorthlighting,
Thanks for the reply. The development site is our internal-only development and testing version of the site that is a near exact copy of the production site. We have never published any urls to the development site (email or otherwise). The production site has been live for over 7 years now, and has been banned in Yahoo for about a year for reasons we still cannot find (watch for this thread soon!).

The two domains are completely separate, (i.e. www.myproductiondomain.com and www.mydevelopmentdomain.com). We do not want the public to see our development site because 1) it is not always stable due to ongoing development and testing, 2) it would be seen as a duplicate site of the production and would be penalized in G and MSN.

My concern is if we open the development version and 301 to the production yahoo will eventually apply the same ban that is on our production site and we will have two banned domains in Yahoo. Another concern is how this will effect our standing with Google and MSN.

Any suggestions?