Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Strange Subdomain Indexing - every page but home page

         

bwnbwn

1:37 pm on Aug 11, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I launched a subdomain about 2 weeks ago. Built site maps, checked robots text took out no index, linked the site from a good source or revelant link. Added to webmaster tools, verified the site everything I can think of and when I check the site every page is indexed but the home page.
Site was indexed within a couple days all but the home page so I figured it would in a few days well 2 weeks latter the home page is still not indexed.

I have PPC as well set up for the site but the keywords I am targeting right now are going to internal pages not the home page.

I am just wondering has anybody else launched a subdomain is having the same issue? I am a little preplexed to this issue and just can't figure out why I can't get the home page indexed.

I have internal pages popping for specific keywords organic that I have PPC going to so now I wonder does PPC give a boost to rankings.

I am going to add some keywords today that will go to the home page and see if there are any changes in the next week.

tedster

5:36 pm on Aug 11, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There are mysterious bugs from time to time that cause index pages to not be in the Google index - and this happens even for the root domain, to say nothing of a subdomain. This seems to me to be most likely if you've already checked the obvious things like robots meta tags and such. If it is a Google data bug, it should clear up within a few days.

Robert Charlton

7:57 pm on Aug 11, 2010 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I have PPC as well set up for the site but the keywords I am targeting right now are going to internal pages not the home page....

This might be a clue.... Back in 2006, Google's Bigdaddy infrastructure began to cache and share data retrieved by various of the Google bots, including Googlebot and the Google ad bots, the Google News bot, etc. Permissions were set up within the infrastructure for each of the services to get pages from this "proxy cache" as appropriate.

Conceivably... and this is conjecture... with the new Caffeine infrastructure (and possibly previously with Bigdaddy), the ad bots might have spidered the pages identified for PPC (you didn't mention whether AdWords or Adsense)... and Google might have indexed the available pages before the organic Googlebot found the index page via the link that you mentioned.

This might explain why you saw "internal pages popping for specific keywords organic that I have PPC going" without seeing the index page. Or, it might be an internal Google glitch involving the above arrangement.

You might have also left a noindex,follow robots meta tag on your home page. It's something I've seen happen often enough now that it's on my check list.

I don't think this has anything to do with a subdomain arrangement per se. It's definitely not favoring pages with PPC.

Matt Cutts has described Google's crawl caching proxy arrangement on his blog, and it's worth a read.

Crawl caching proxy
April 23, 2006
http://www.mattcutts.com/blog/crawl-caching-proxy/ [mattcutts.com]

bwnbwn

8:24 pm on Aug 19, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have found the issue or non issue. The index page is now showing in the serps although still not catched.
In early July Google tried to spider the site and hit a no index robots.txt no archive meta tag as it should since the site wasn't completed and I didn't want it indexed, but seeing how the robots.txt file has been downloaded many times after I removed the no index it just seems slow to respond.
I have looked even pinged the page many times and there wasn't any message in webmaster tools robots.txt area until Tuesday.
Seems updating webmaster tools is 2-3 weeks behind.