Third Link Level Not Indexed by Google

[WebWare]

Lately I have noteced that the google did not indexed the third link level at my site which is the most important aspect in my site, So what is this problem guys.
Thanks

[ciml]

Google's depth of crawl on a site depends mostly on the site navigation and the *PageRank. If, as is often the case, PageRank is fed mostly into the home page, then it's important to let it flow cleanly to the bottom levels.

It takes more PageRank for a large site to be indexed fully, so using clean URLs (eg. no "?" characters") is important. Often, it's useful to 'promote' the most important bottom level pages up a level. This helps Googlebot (and human users) find them more easily. Sometimes, a 'specials' or 'most popular' link on the home page is appropriate.

* I'm not clear on whether Googlebot is programmed to follow PageRank, or whether Googlebot's crawling and PageRank's 'random surfer' model just happen to coincide as one would expect.

[johnser]

I've had a similar experience

New ecommerce site launched 2 months ago has 300 static html pages and 3,000 template driven pages.

ie - 3,300 pages to crawl - link structure was ok I thought with each of the 3,000 dynamic pages being well-linked to.

However only the main 300 pages have been crawled.
Looks like I should get some PR...

Any other way to encourage Gbot via on-page linking structures?

J

[martinibuster]

ciml posted some good advice. The next step to take can be setting up site maps- this has the effect of increasing the amount of links inbounding to those deeper pages.

Very important however is to organize the site map in a manner readily usable for the user, that it makes sense. If it makes sense to the user then it'll make sense to the bot.

I enjoy making site maps.

[johnser]

Thx martinibuster

Thing is though, theres so many pages on the site that a static list of 200 urls is only 2 clicks from the home page and I can't get these 200 urls crawled.

I agree that site maps are vital (& easy to do) but I just don't think it will help here because the link structure is exactly the same as the most perfect sitemap I could wish for.

Am a bit lost :(
J

[WebWare]

ciml,There is no choice to omit "?" .

[ciml]

You can get good listings with a "?" in the URL; even one of the top 10 for 'download' has a "?". However, the Shockwave download page has over 20,000 links, including high PR places like NASA, and I gues that neither you nor I have that kind of linkmap footprint.

"?" characters are not mandatory in the URLs of any World Wide Web context I'm aware of. I would hassle the server vendor, hosting company, programmer, CMS creater or whoever is responsible to get it changed. Currently, I have two programmers working on applications that normally have "?" characters, making them run smoothly without the "?" regardless of cookies and Javascript support in the browser. A little costly, but worthwhile.

If you don't have the PR to overcome the slight "?" barrier to getting crawled; then IMO you need more PageRank.

[WebWare]

How can I get more page rank for the third or even the fourth leve?!

[menton]

Hi WebWare,

Have you got a good sitemap? This would be the best way.

menton

[takagi]

How can I get more page rank for the third or even the fourth leve?!

Get links from other sites. The higher the PR on the page linking to you the better, unless you have to share that link with a lot of links to pages on other sites.

[bilalak]

I suggest that u emulate those template driven pages into simpler html pages.

I am using this to let most crawlers see my internal pages.

If you are on apache use .htaccess
if you are on IIS then move to apache ;). There is a tool for IIS but it is not as valuable as mod_rewrite

Then add some listing pages to those template driven pages and add the URL of the listing pages to your site map.

So you have this structure

sitemap
listing pages (with navigation and html url if possible)
emulated template driven pages into simple html address

Luck!

[DaveN]

I get more page rank for the third

Deep linking is the easiest, get a site to link into the third and Fourth levels.

Dave

[ciml]

menton's suggestion is fine if you have up to 100 pages to link (otherwise you should split the sitemap into a number of pages). Personally, I prefer just to use the theme pyramid site structure.

takagi is right about just getting more links; the PR should flow around the site (assuming that there's enough of it to flow). DaveN's suggestion will do more than give you PR, those links coming straight into the most valuable deep pages are worth more than the PR benefit the deep pages would get via the home page.

[martinibuster]

menton's suggestion...

Menton is a day late to this party. Menton should take a moment to read the previous posts to make sure Menton isn't repeating something that has already been said, as I already suggested a site map in message#4.

Jeez...