Google only followed 2 links, requested dynamic URL w/out pararameters

Google did not follow all my links/ Did no request url with parameters...

[JoJoH]

Hi all,

I've just read my log files and it showed that Google had visited and requested pages from my new site. My site is a dynamic site, that means almost every page on my site is dynamicly generated(with the "?" and parameter in my URL). In my log files, it showed that

1)Google requested my home page, then followed only 1 or 2 out of like 10 links.. Why?

2)Now say I have a dynamic page called ww.mysite.com/somepage.asp. That URL changes based on the user's selection. For example it could look like that: ww.mysite.com/somepage.asp?cat=widget. Now in my log files, it only showed that Google requested this ww.mysite.com/somepage.asp without the parameter! Well that is not good cuz my product catalogs are generated dynamically which means if Google does not index the full URL with the parameter than none of my product pages will be indexed! Please advice...

Thanks in advance!

[WebGuerrilla]

Do your logs show the paramaters for visitors other than Google? Some servers are setup to not record the parameters in the log file by default.

I've run into this problem a few times with clients running on MS servers.

[JoJoH]

Thanks for the reply WebGuerrilla! Yes, my log files do show visitors other than Google. But none of them display the parameter of the URL... also, my web host runs on Win2000 servers...Please advice.

Thanks in advance.

[johnser]

I think you need a certain value of PR before the spider will crawl deeply into a dynamic site. This is what I'm working on at the moment to overcome.
J

[ogletree]

Make a site map. That works wonders.

[Herenvardo]

You must know that when you are using dinamic links you are making a bet: you can offer better access to the information in your web to your users, but your page could be "unspiderable". The option of a sitemap is a good chance to prevent this problem, even sometimes it could fail. In any case, a sitemap is always a good option, both for visitors and spiders.

[skipfactor]

I've added 3 dynamic sites in the last 5 months with varying degrees of initial PR(0-4), and my opinion is that it just takes the bot some time to get in the mood, but of course the higher PR sites seemed to be hit deeper sooner.

I wouldn't worry about it grabbing the URL without the parameter. I noticed this early-on, but the URL wasn't indexed as such.

Here [webmasterworld.com] are some observations I made on 2 of these sites comparing the number of digits used in the id= that might be of help to you.

[JoJoH]

Johnster: Thanks for the reply! Anything idea in particular regarding to increasing PR?

Please advice. Thanks again for your help!

Ogletree, Herenvardo: Thanks for the reply! Yes I do I site map but:

1)Google did not follow the link from my home page's or any other pages' link to the site map at all.

2)Majority of my site map are also dynamically generated, any negative effects?

Please advice. Thanks again for your help!

Skipfactor: Thanks for the reply! Interesting study you've got there! My productIDs ranges from 1 to 500s so I hope Google will come and pick them up soon. So you are saying that it is normal to not see the URL parameter in the log files? if yes then it is a big relief to me! Thx. :) As for increasing PR, an good ideas in particular?

Please advice. Thanks again for your help!

[ogletree]

I have a PR2 site and I have no URL's that have pages all my URL's are www.domain.com/?Pane_2=body-subject-blah_blah I have no .asp .html .htm or anything. (That URL is generic for my CMS and not specific to me so don't snip it) Most of my site is in Google. It happened pretty quick. I use a DHTML menu that Google does not even look at. Everything comes from my site map. The funny thing is that I have some pages that are indexed and some that are not. Even when the links above and below it on the sitemap is included. My site map is a little long I plan to split it up into 2 or 3 pages instead of one. Make sure you only have 100 links per page.

[Herenvardo]

JoJoH: Please read these questions:

1 Is the link from your homepage to your sitemap something like: <a href="/sitemap.html>Sitemap</a>?

2 Is your homepage standard HTML, without frames o with a good NOFRAMES tag?

If your answer to any of this questions is NO, you know what to do. But if you can answer YES to both questions, then I'm very surprised. In this case, i'm not sure from where your problem comes, but it's not about the dinamical links, nor parameters. If you link standardly from a standard homepage to a standard sitemap, the link should be spidered by gbot.

[WebGuerrilla]

JoJoH,

Your entire site is showing PR0. Since you mentioned it was a new site, I wouldn't worry much. More than likely, you just need to wait it out.

However, you might want to reconsider your file name structure. The urls are a bit extreme. If you are going to keep them that long, you might want to consider hyphens instead of underscores.

[JoJoH]

WebGuerrilla: Thanks for the reply! Actually when I said my site is new that menas that I've just optimized it/redesigned it. Here is what happened. My site-the basic design was on the web about 6 months ago, I didn't optimize it or anything... somehow it got into Google, probably from someone who links to my site. Not until recently, I have time to optimize and totally redesigned my site. I am currently waiting for Google to realize that my old site/pages no longer exists, I have a new site for it to index... So far all my old pages are still "stuck" inside Google's system, my new pages are still not yet inside it's datacentre... wonder how long do I have to wait. Regarding the file names, at first(I read it somewhere)I thought keyword-centric file names are going to improve my rankings, but many people have told me that they won't do any good... so I guess I should probably renamed them to shorter names. Anyways, how did you check my page rank? Did you just type in a keyword in google and check on it using the Google toolbar? Please advice. Thanks in advance!

Herenvardo: Thanks for the reply! The link to my site map is just any regular HTML links. My home page has no frames, no javascript. Just HTML, stylesheet, and VBscript.

Ogletree: Thanks for the reply! My site map has and other pages has less than 100 links... Don't understand Google alot of times, I guess I would just wait patiently...
Anyways, how did you check your page rank? Is it my typing in your targeted keyword in Google and look at the Page rank is Google's toolbar? Please advice. Thanks in advance!

[ogletree]

I am not sure what my actual PR is but the toolbar says PR2 The directory says it is PR2. It was a PR3 for a short period after esmerelda. The funny thing is that it is number one for a keyword and it is beating out a PR3 so I am sure it is a PR2 or PR3 not sure exactly.