New Site, Googlebot Not Crawling

Reads robots.txt and default page but no more.

[MikeeG]

Hi

Apologies if there are many posts like this, but I have a problem in that I have had a new site running to 2 months now. Looking at the log files, Googlebot looks everyday at my robots.txt file and default page but no more?

I've checked the robots.txt file and it fine, I've even tried deleting it totally.

Default page has meta tage of:-
<meta name="robots" content="index,follow">

Am I just being impatient after just 2 months? or should Google had done a deep crawl by now? (I've been checking the logs every day) I also have a few links on other sites.

Any advice would be appreciated.

Thanks

[teeceo]

I had the same prob. intil yesterday...(index page was in but nothing else) what I think you should do is get like 2 or 3 links goinng to (non-index pages) so that you get more exposher. Thats not want worked for me but I think it couldn't hurt. in my case googlebot just added the other (very few) pages becouse it wanted to i guess.......Later.

teeceo.

[pagano]

I have the same problem too; a client site with lots of unique and valuable content (product-pages with extended description, references pages, some pages with a lot of text about the technology that my client uses), it has also some good inbound links (1 of them is a DMOZ link from a category with PR6 ...). I have not only optimized each of the pages, but also put up a sitemap page (with standard HREF links), directly linked by default page.
Results: googlebot continue spidering only robots.txt and default.asp, no any other pages.

ALSO: if I search on GG with my client's URL I see in the SERP the listing with:
"Category: World > ..." which is the PR6 DMOZ category mentioned above, and that should mean that GG has found that important link, BUT if I search for "link:www.myclient.com", the result is: "Your search ... did not match any documents"!

Any help, please?!
Many thanks in advance

[Dayo_UK]

pagano

Check the cache of the Dmoz page - is your site in the cache?

If so then with a Dmoz link it should not take to long to be indexed (although Google appears to be having problems indexing Dmoz too.)

And of course link build - however Google may treat this as spam, and yet reward it at the same time depending on its mood.

I think I am on the brink of giving up! - Monday morning feeling perhaps.

[keeper]

Any other search engines pick up your pages?

I've noticed Googlebot nibbles before he bites.

[petehall]

We seem to be having the reverse problem, whereby Google refuses to drop old pages. The old pages redirect to a 404 error page yet they are still appearing on relevant searches.

[experienced]

Do you have backlinks for your site..?

I Uploaded my 4 Stes 10 days back with 4/5 the good backlinks and fortunatelly i have got all of them indexed with all the pages.

Check your Backlinks. HTH

Thanks
Exp...

[pagano]

Dayo_UK

first of all, thank you; then I check cached version of DMOZ category page, and there I can find my client's backlink, so I think that something should come at next update.
But ... I return to the SERP for "www.myclient.com", and I re-check category link under the listing, "Category: World > ...", which links to the Google version of the category (http://directory.google.com/Top/World/ ...)

The nice thing is that in the Google version of the category, the backlink to my client site DOESN'T appear!

So, I could say: why google put a link under my listing to a page where does not appear the listing to my client's site?

It seems that, in building SERPs, GG considers a directory version which is not the one we see when browsing directory.google.com

Does it seems something meaningful?

[jady]

We launched a new site a month ago, upon launch posted about 10 good off-site inbound links to it. No more than 3 days later the site was fully crawled and actually ranking well. Then it dropped back out. I hear this is normal - kinda G's teaser.

Bottom line, I feel the inbound links are what you need to establish a relationship with that site and Google.

P.S. I never used the free submit on Google.. :)