Google Indexes New Domains - from Registar Information?

[incrediBILL]

I've often thought this was the case and maybe it's been discussed her before but I just did a recent experiment to prove it.

Just registered a new domain with GoDaddy about 2 weeks ago and published a temporary site the same night.

I haven't written about the domain, given it to anyone, the pages don't have any links to anything like Google Analytics nor AdSense, it's a plain vanilla HTML temporary site.

I checked Google's index a couple of days later and VOILA! it's already in the index and Google claims ZERO inbound links.

The only conclusion I can draw from this is that Google uses their registrar status to find and index new sites the minute they're posted the first time.

Bing and Yahoo - nothing.

So for those of you worrying about how to get into Google's index, it's not so hard apparently, just register a new domain! ;)

[jmccormac]

Parsing the zonefile will allow them to detect new domains and websites. It is actually that simple. However it does open the index up to pollution and gaming.

Regards...jmcc

[ken_b]

I registered a new name a couple weeks ago and after reading this thread I did a Google search for the domain name .com here's what I got at #1 ...

Business profile for example.com provided by Network Solutions
Learn about example.com from this business profile provided by Network Solutions.
example.com cache

Notice the "provided by" part.

Beyond registering them I have done nothing with these domains, nothing, nada, zip. Any coming soon, profile, etc, is not my doing, as far as I know.

Same shows in Yahoo, at #1, but not in Bing.

Should I be thanking NS?

Does GoDaddy do something similar?

[Sgt_Kickaxe]

The only conclusion I can draw from this is that Google uses their registrar status to find and index new sites the minute they're posted the first time.

A more likely conclusion: There are dozens of fairly spammy sites that report new domain name registrations. The spam sites hope to rank well for whatever your domain name is at least temporarily. They are of low value in the food chain but if someone is looking for your domain name online they show up... for a while. It's worse with phone numbers or IP address lists but new domain registrations has become spam fodder too. It is impossible NOT to have your site mentioned online when you register it.

Bing and Yahoo - nothing.

They're crawl and index speed is much slower.

I checked Google's index a couple of days later and VOILA! it's already in the index and Google claims ZERO inbound links

Spammy sites don't count as inbound links, you will rarely see them even in Yahoo! but they are crawled anyway.

just register a new domain! ;)

and pray the spam finds you! :-)
if it doesn't just head over to a site metrics SEO ranking reporting site or three and look up your own domain... that should give you just enough spam power to get on radar. There is value in spam sites afterall... too bad you can't shake them when you're done using them.

[incrediBILL]

There are dozens of fairly spammy sites that report new domain name registrations.

I don't think that's the case this time because I'm well aware of those spammy sites and Google seemed to index this new domain way too fast to find it trolling those domain dumps.

Plus, I checked a couple of the more notorious sites and nothing showed up.

[bwnbwn]

got the google tool bar installed this is another way the site will get crawled and indexed but your right on the register as it makes sense to crawl them and take a quick peek at the content going up.

Standing deal here any new site gets an automatic no index right away in the tag and robots txt file.

[TheMadScientist]

Actually, this is something I noticed with a tld Google is not a registrar for, and they have some type of affiliation with GoDaddy (I don't remember what it is off the top of my head, but I know when I was looking into it there was 'something' there) and personally I think GD is sharing registration data with G, because the domain was spidered within 2 hours of registration.

[jmccormac]

There is a fast zonefile update service available that publishes new additions and modifications to the zonefiles. It is a value added service from some registries. The zonefiles themselves are updated typically on twelve hour cycles. How soon after the domain was registered did Google's spiders hit the site?

Regards...jmcc

[TheMadScientist]

If you're asking exactly on mine, I know it was less than 2 hours before the bot visit (it was not in the index, but it was visited), because I have a DB table full of bot visits and was shocked to see GBot had already visited the site when I clicked refresh on it, and much like incrediBILL NO ONE knew I registered the domain at the time. I mean it was a 'spur of the moment, didn't even know I was going to do it myself until I did' registration, so there were absolutely no links, I haven't used the tool bar in years, G is not a registrar for the tld, it either had to be GD or the DNS system like you're talking about, because there's no other way they should have known it was registered... I hadn't even visited the site and the GBot visit was almost immediately after I uploaded the standard 'I'm going to do something here someday' page I have that records visits.

[jmccormac]

It sounds like they are using the update service to detect new sites. It is a theory of mine that Google's famous link based system has a major flaw in that many new websites don't have inbound links and don't develop them for a while. This may have forced Google to look at new (or relatively old) methods of site detection. It might be interesting to test it with a non-GD registrar. It is easier to handle the smaller zonefiles than it is to handle the .com zonefile (.com expands to about 6.5G of data) so they could have been using zonefiles and the domain registration time could have been near to one of the zonefile updates. A ccTLD registration test may help confirm if Google is using an update or zonefile service because most ccTLDs will not provide external zonefile access. This means that Google has to detect new ccTLD sites by crawling and if they pick up an unlinked ccTLD site as quickly, they could be using another method of detection.

Regards...jmcc

[dstiles]

I found this out two or thee years ago! New domain, set up IP, set up root of site and before I'd uploaded the site the logs showed google. This was a domain registered with tucows.

I don't think it can happen (so easily?) with UK/Nominet domains but given the trashy management of US managed domains (com/net/org) I'd say it would be surprising if it were not possible.

[leadegroot]

Hosting on WHM (Cpanel) means you can't create a site/domain until it exists (has been registered)
When using alternate hosting i used to upload the site before the registration and it was live immediately.
Its a annoying difference because I often saw Googlebot hits in the logs not long after registration - they are obviously able to take advantage of their registrar access because it *never* happpened with .au domain registrations, for which they are not accredited.
YMMV
(Note that this didn't get me indexed, that I recall - its been over a year - but I really liked seeing that first crawl.)

[nethead]

I've seen that happen for the last 2 years but recently more common. I've set up a site and within max 2 days you can find it in google's index. Its not hard but ranking for them now is a different story...

Sometimes I see SEO companies or pro's making staements like "I can get you in Google" and it makes me laugh since all you need to do is register a name and add a paragraph and at times not even that...lol

[mhansen]

I think the toolbar is more of a window than a toolbar.

The only way that I have found to really keep googlebot out of a website is to not have a website!

[indyank]

Yes google does discover new sites pretty quickly these days.There definitely seems to be a strong link between google and domain registrars. Probably google feeds a lot more info into its database during this process, like all sites owned by the registering user etc.

[dstiles]

Wonder if they also get the "private" registration details when the doman is "protected" - ie phone, email etc not displayable. And if so do they match it to other domains.

Just a thought. :)

[Lorel]

I load new sites being designed in a folder on one of my sites with noindex and disallowed in robots txt, then everything goes online all at once.