Welcome to WebmasterWorld Guest from

Message Too Old, No Replies

Google Only Scratching the Surface



3:28 pm on Jun 22, 2005 (gmt 0)

10+ Year Member

After almost all my pages went url only, google has re-indexed about 260 of my 3400 pages over the past week. What I have noticed is, the pages that were indexed are only the ones that are on the surface. No deep crawling has occured.

I was thinking that since google won't deep crawl my site, maybe I should create a page that's linked to from my home page that lists the links to all of the pages that need indexing. This will be quite a long page with a couple thousand urls on it. I'm wondering if google will index all these pages given that they don't require any deep crawling to find.


8:59 pm on Jun 22, 2005 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

You should have a site map page, maybe even several - each organised a section at a time.

Google will spider a file up to at least 100K in size, but at the same time advise using less than "100 links per page".


9:22 pm on Jun 22, 2005 (gmt 0)

WebmasterWorld Senior Member bigdave is a WebmasterWorld Top Contributor of All Time 10+ Year Member

sitemap pages help, but don't expect instantaneous results. 260 pages coming back up in a week is doing pretty good. You would probbly see consistent improvements even without sitemap pages.


11:27 am on Jun 24, 2005 (gmt 0)

10+ Year Member

If it's possible for you to get sitemap(s) up then it can only help. An other option is to get some links from other sites linking to some of your deeper pages.

On a similiar note - I've launched a new site (perhaps 2 weeks ago). So far google is only showing about 2 pages for it...... but looking at my stats I see they have been visiting many times of the last few days and must have several hundred pages at least. How long does it usually take these days for these pages to be included? or does anyone know if these pages may actually be available in some datacenters?
I would have asked this in a different post, but they usually never get validated... perhaps its just me ;)



Featured Threads

Hot Threads This Week

Hot Threads This Month