Forum Moderators: Robert Charlton & goodroi
My site is over year old, LOTS of backlinks, ranks well in Google for terms, PR4 etc but all my pages dont seem to get indexed.
For example I have a linkdump on my main page that uses archives, only about 4 of the archived pages are indexed, why doesnt it index the others? Backlinks isnt a problem, pr isnt a problem, it isnt a new site, surely it should deep crawl the site and index them? It also only seems to cache my other pages once every ten days or so, some havent been cached since last month but ALL pages are linked nicely so everytime the bot will come to my site (Which is daily) it will easily find the other links, the navigation system is fine.
Any ideas?
Thanks
I've been in this position often, and truth be told, it's a rare site that is ever "completely" indexed. This is true for sites at all scales, from the very modest up to the million-url monsters.
Plus, in recent weeks Google has been quite volatile and not showing a normal pattern at all. If the history of previous years holds true, "normal" will not reappear until the firt full week in September or so.
It's tempting to think of Google as an application that spiders and indexes the web - but it's really a BUSINESS that spiders and indexes the web for its own business purposes. There are all kinds of technical and business logic built into their indexing programs that we may never comprehend from our position.
That said, I would suggest you make sure that the total number of links on your home page stays modest. Under 100 is a good rule of thumb, and I would go for as much lower as you can work with practically.
It's tempting to think of Google as an application that spiders and indexes the web - but it's really a BUSINESS that spiders and indexes the web for its own business purposes. There are all kinds of technical and business logic built into their indexing programs that we may never comprehend from our position.
I think I know where you're going on this, but I also think it might take me a couple of days to really wrap my head around it. Any chance you might devote a thread to it and expound a bit?
