lucy24 - 10:07 pm on Oct 29, 2013 (gmt 0)
For comparison purposes I did a site: search on my personal domain-- with search field itself left blank-- to see what comes up. There are 8 main directories. (Actually more, but only 8 are linked and indexed.) Result:
#1 is www.example.com/ This has to be an artifact of the site: search process, as I'm not a front-driven site and nobody ever goes to the front page. Until a couple years ago I didn't even have a front page.
We then proceed to
Still not significant pages-- I did say I'm not front-driven, but you thought I was exaggerating-- but in recent months the googlebot has been crawling these two index pages at almost bing-like rates. And they do lead to most pages that humans actually visit.
skipping over /dir8/ itself. /subdir1/ has been getting a lot of traffic recently (I suspect one of the English universities is teaching a course on its subject). Nobody ever goes to /subdir2/, but structurally it's almost identical to /subdir1/.
/dir8/ in its own right doesn't show up until much further, after most subdirectories of /dir4/.
Interesting detail here. The directory I'm calling /dir4/ currently leads on to 15 pages, of which 13 are /subdir/ index pages. One of those 13 is a terminal page, with content. This is the only one that doesn't float to the top in a site: search. The other 12 are gallery-type pages that lead on to individual content pages. The remaining 2 are galleries that look exactly like the 12 /subdir/ pages to humans; only a search engine-- or a human who looks at their browser's address bar-- can see that they're structurally different.
A final quirk is that about half of those /subdir/ pages lead only to noindexed pages. The site: search doesn't seem to care; the /subdir/ pages are listed in more-or-less random order.
At the higher level, only /dir3/ is a content page. The others are all indexes. (I don't here mean auto-generated index, but a page whose primary purpose is to link to everything else in the directory. So not much content.) In a site: search, this fact is enough to outweigh all but the superattractions of /dir1/ and /dir2/ -- the exact opposite of the behavior I see in the subdirectories of /dir4/.
Tentative conclusion: A site: search is more interested in structure than in content.