homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

Google having trouble deep crawling my lowest level pages

 4:50 am on Jul 18, 2004 (gmt 0)


I am having issues with google consistently indexing ALL of my pages in my lowest directories.

The question I want to ask is CAN I IMPROVE MY INTER - LINKING to get google to go deeper into my site more easily?

Currently my site looks like this

index->main nav(category 1) ->category 2 ->lowest level(1000 pages)

My index and main nav are cross linked to each other, EVERY Page points to Index and my main navigation links. After that there is just 1 link to Category 2 and from there 1 Link to my products.

If i have the links in category 2 cross-linked to each other would that improve things. Right now my site looks like this

index <--links to--> Accessories(nav) --links to--> Handbags(20 items)
index <--links to--> Accessories(nav) --links to--> Wallets (10 items)
index <--links to--> Accessories(nav) --links to--> Coats (10 items)

Hand bags, Wallets, and Coats don't point to each other, just to the index and main navigation. The products only point to index, main navigation and back.

Would it also help to add on each product page a PREVIOUS PRODUCT and NEXT PRODUCT link so that each product page can have at least 3 links pointing to it? right now for every product there is only 1 link?

Google got all 200 of my items last month. A test This month half my items got dropped. I get a bunch of search results that look like this:

similar pages

similar pages

similar pages

The other half got updated. Does this mean it can't find the half that got dropped and now only appearing as

similar pages

I was told that google does this if it has persistent problems getting to a page.

Is this links issue, or does it more have to do with my web hosting and my pages not being available?

Yeah I plan to put on 1000's of products. I was starting to get some traffic and then half of it dissappeared with my pages. Yikes.

IF ANYONE IS INTERESTED all these sites are in PHP and google TAKES A LOT LONGER to index them than a URL that looks like STRAIGHT HTML. that is my experience now.




 2:22 am on Jul 19, 2004 (gmt 0)

1) mod_rewrite will help you
2) If you've got a pyramid

Index -- cat 1 - sub cat 1 -- hundreds of items
- sub cat 2 -- hundreds of items

Try to create best seller lists on cat 1 and sub cat 1 level which link deep into the best sellers.

2) Also try to get inbounds into cat 1 and sub cat 2 levels.


 12:29 pm on Jul 19, 2004 (gmt 0)

IF ANYONE IS INTERESTED all these sites are in PHP and google TAKES A LOT LONGER to index them than a URL that looks like STRAIGHT HTML. that is my experience now.

To clarify, google will not apply such high PR to pages which have urls with lots of arguments (page.php?something=somethingelse&bobbins=monkey )

IMHO, google doesn't have a 'problem' spidering anything, it's simply a matter of what they consider to be 'authoritative'.
<edit>Part of this is because Google try not to kill script/database generated sites by getting the googlebot to go 'easy' (i.e. not aggressive) on pages which look like they are generated by a database.</edit>

Using mod_rewrite (if your using Apache), will allow you to use more 'standard' url schemes, while retaining your php scripts (page/something/somethingelse/bobbins/monkey) for example..


 1:43 pm on Jul 19, 2004 (gmt 0)


that is what I think is happening too. I am having problems with urls that have parameters in them. I am thinking that google has a short time-out for these pages, which agrees with what you are saying. I will use mod rewrite, hopefully this will take care of my problem. thanks guys

stevenb 1959

 2:11 pm on Jul 19, 2004 (gmt 0)

Have good link structure by moving your page rank down through your site so the spider will follow.
Next you need a link or two on topic (number one in Google for their search phrase) coming into your website at the top and at the bottom of your website, both PR4 and PR5, and your webpages will be indexed by Google every 4 weeks.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved