Forum Moderators: open

Message Too Old, No Replies

How are dynamic pages indexed?

?!?

         

Beastie

12:17 pm on Apr 20, 2003 (gmt 0)

10+ Year Member



I have a couple of questions that have been bugging me for a while:

1. If a page is dynamically generated, how does it get crawled? The page does not exist in a specific state until the query string is passed to it.

So, what if PageA.asp has twenty different query strings (options) which generate twenty different versions of pageB.asp? Does Google send twenty seperate requests? Do the SERPs reflect 20 different versions of pageB.asp?

2. What about PR being passed on from search engines where results pages are generated dynamically? The SERPs for MSN have a PR8 with 18 listed sites on each page. Can PR be passed on from these pages? if so, how? Again, these pages do not exist until a specific search term query is passed.

The confusing thing is that SERPs on all search engines have PR values. Does this imply that Google has crawled the actual page of results, or simply that it has crawled the filenames which generate those results?

takagi

12:35 pm on Apr 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



1. If a page is dynamically generated, how does it get crawled? The page does not exist in a specific state until the query string is passed to it.

Google will 'see' the same bytes as your browser does. Just make sure there are links to all the different pages. Google will spider them and assign each an individual PageRank. So it is possible that one query has PR3 and another has PR0, but it is the same PageA.asp

2. What about PR being passed on from search engines where results pages are generated dynamically? The SERPs for MSN have a PR8 with 18 listed sites on each page. Can PR be passed on from these pages? if so, how? Again, these pages do not exist until a specific search term query is passed.

The PR8 is a guessed PR. Google doesn't know the page, so it will assign it a PR that is 1 lower than the PageRank of the directory above. Links from a page with a guessed PR won't influence the PR of the page linked to.

Beastie

1:27 pm on Apr 20, 2003 (gmt 0)

10+ Year Member



So, would pageB.asp appear 20 different times in the SERPs, or just one time? If the latter, which query-string generated version of pageB.asp would be indexed?

If pageB.asp is *not* indexed 20 times, then can I assume it is a huge advantage to have the content on static html pages instead of the asp ones? this would ensure all the pages are in fact indexed.

Dreamquick

1:42 pm on Apr 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Beastie,

I know for a fact that if pages are sufficiently different each "version" of the page will be indexed separately - however I don't have experience of what happens if the pages are very similar.

The thing to be aware of is that too much querystring data will make search engines very wary of crawling the page because.

- Tony

takagi

1:54 pm on Apr 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I don't understand what you mean with PageA.asp and PageB.asp, but let me give an example.

Suppose there is a site which has information about 20 countries and this information is stored in a database. This information can be accessed through PageA.asp like this:
www.mydomain.com/tourism/PageA.asp?country=1
www.mydomain.com/tourism/PageA.asp?country=2
:
www.mydomain.com/tourism/PageA.asp?country=20

In this case, Google could index 20 different pages. Each with an individual PR, individual cache, individual backward links etc. If all have a link except for country 5, Google will index 19 pages. G will not search for a missing page. When searching for

site:www.mydomain.com inurl:PageA.asp

you can see the 19 indexed pages. If country 7 has an external link, then the PR for the page of that country is likely to be higher than the other 18.

I hope this makes it clear to you.

Beastie

1:58 pm on Apr 20, 2003 (gmt 0)

10+ Year Member



Clear as a whistle now. Thanks everyone for helping me!