| 4:24 pm on May 25, 2010 (gmt 0)|
|I've created a GWT account, which suggests there is very little wrong with it. |
That seems to indicate that at least some URLs are being indexed - right? Webmaster Tools only has data for indexed URLs, so you might mean there's a low level of indexing, or that the site: operator number is bad, or...
| 6:15 pm on May 25, 2010 (gmt 0)|
Sorry Tedster - should have been more specific. The site definitely isn't indexed - you won't find it in Google by doing a cache: search, a site search, a search for the domain name, a Toolbar cache, or if you search for a snippet of homepage text. It's definitely not there.
Weird thing is, server logs are showing Googlebot accessing the website and seeing 200 status codes. Even crawl stats in GWT is showing some (albeit small) level of Googlebot activity on the site. If I fetch as Googlebot in GWT then everything works fine. Other than that, there's no data available within GWT.
| 6:40 pm on May 25, 2010 (gmt 0)|
I've had 2 sites not get indexed for over a month now. However, 11 other sites launched at the same time have all been indexed fine and are even ranking for various long tails.
I can't tell the difference between the 2 that are not being indexed and the 11 that have, other than the 2 that haven't been indexed have many more backlinks ... which would suggest faster indexing wouldn't it? It doesn't make any sense.
All 13 sites had noindex tags for about 2 months whilst being developed. The noindex tags where all removed at the same time.
| 7:50 pm on May 25, 2010 (gmt 0)|
@internetheaven - thanks for the reply. the subdomain that i'm having problems with initially had a noindex tag, and all links to it were nofollows. We dropped these though when wanted to get the sites indexed. I'm wondering whether there's a bug with Google whereby if a link / page was nofollowed / noindexed, and that gets changed, then perhaps Google keeps the old nofollow / noindex status assigned to it
| 8:35 pm on May 25, 2010 (gmt 0)|
To clarify - in addition to a robots.txt disallow directive that you mentioned above, did you also use either a robots meta-tag nofollow or a nofollow attribute at the individual link level?
| 8:53 pm on May 25, 2010 (gmt 0)|
@internetheaven - I've had the same thing with a site not being indexed. Launched 2 sites about 2months ago, indexed fine, then another about 4wks ago, still not showing up. No reason why it shouldn't be, content is good, links in moderation etc. Should be fine, but still not appearing.
| 6:10 am on May 26, 2010 (gmt 0)|
id experience this one before. When i created forum (simple machines).
It takes 4 weeks to get indexed... it was kinda weird.
| 2:18 pm on May 26, 2010 (gmt 0)|
@tedster - it was a page level noindex, follow meta tag, and external links from our main domain were nofollowed. Both have now been removed
| 5:54 pm on May 26, 2010 (gmt 0)|
That means a lot of pages need to be re-crawled - and possibly more than once, to confirm - before they are included in the index. Four weeks does seem just a bit slow, but it's not outside other's experience.
| 10:59 am on Jul 1, 2010 (gmt 0)|
Since I last posted here, the domain has still not been indexed, despite being crawled by Google. Any other ideas?
| 12:42 pm on Jul 1, 2010 (gmt 0)|
Does that mean that google crawls a complete site before the pages start to showe up in the index ?
| 2:35 pm on Jul 1, 2010 (gmt 0)|
@centime - I don't think that should be the case. Google should index the pages as it finds them - no need to crawl the whole site before it indexes it. There's definitely something really strange happening with the domain
| 5:43 pm on Jul 1, 2010 (gmt 0)|
|@tedster Does that mean that google crawls a complete site before the pages start to showe up in the index ? |
No that has never been the case.
|There's definitely something really strange happening with the domain |
Sure sounds like it. This is a subdomain, correct? Is it on a dedicated IP?
I've never personally taken an indexing problem to Google's webmaster support forum. But in this kind of situation, I would at least consider it.