homepage Welcome to WebmasterWorld Guest from 184.73.52.98
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Dynamic pages
Will they get spidered and listed
jjdesigns4u




msg:77565
 10:10 pm on Feb 11, 2003 (gmt 0)

Will Google see my dynamic pages?

Does Google ingnore the link below because it has a?
mysite.com/_shopByBrand.aspx?brand=Body%20Tone

What is the final or current say on dynamic pages and gettting them spidered and listed?

I thought Google did not follow or list dynamic pages?

Thanks

 

Krapulator




msg:77566
 10:13 pm on Feb 11, 2003 (gmt 0)

Hi There,

If you keep your querystrings to a maximum of two short variables, you should be OK.

jjdesigns4u




msg:77567
 10:17 pm on Feb 11, 2003 (gmt 0)

2 variables after the? so the one above is to long.... right?

What about the link? I it linked from a drop down thats bad right?

What if it was a hard link to that page?

Thanks

DerekT




msg:77568
 10:54 pm on Feb 11, 2003 (gmt 0)

Actually that url has only one variable and should get spidered.

mysite.com/_shopByBrand.aspx?brand=Body%20Tone

A breakdown of brand=Body%20Tone :

Variable = brand
Contents = Body%20Tone

I would ditch the %20 in your strings and use something like Body-Tone or Body+Tone but that is just a suggestion. You could also ditch the variable strings and clean up your URLs with mod_rewrite.

www.site.com/brand/Body-Tone.html


RewriteRule ^brand\/(.*).html /_shopByBrand.aspx?brand=$1 [L]

atadams




msg:77569
 11:00 pm on Feb 11, 2003 (gmt 0)

Agreed. This URL should be fine. Although I would also get rid of the URL encoded space (the %20). I replace spaces with "+" (I use the + only because that's what Google uses) and it works fine.

jjdesigns4u




msg:77570
 11:49 pm on Feb 11, 2003 (gmt 0)

OK.......

So My asp site with dynamic pages will have no problem getting spidered and listed.

So I should optimized these pages.

What about the linking anything I should know

taxpod




msg:77571
 3:19 am on Feb 12, 2003 (gmt 0)

That's right. I get tons of pages .asp?variable=whatever included.

Sammy Jankis




msg:77572
 10:16 pm on Feb 18, 2003 (gmt 0)

I have been told that instead of using mod_rewrite, you can use something similar if you are running on IIS. Can anyone confirm/ elaborate on this?

paladin




msg:77573
 11:54 pm on Feb 18, 2003 (gmt 0)

I don't know about the mod_rewrite, but it should be pretty easy to create a custom 404 file that will do something similar. I can't remember where, but I have seen quite a few ASP based forums that do this.

Alon

I have been told that instead of using mod_rewrite, you can use something similar if you are running on IIS. Can anyone confirm/ elaborate on this?

Sammy Jankis




msg:77574
 12:04 am on Feb 19, 2003 (gmt 0)

Just to clarify Paladin, are you talking about a method of changing the url so that it doesn't have a query string in it? ie. (using DerekT's example)

mysite.com/_shopByBrand.aspx?brand=Body%20Tone

could become

www.site.com/brand/Body-Tone.html

My original message might have been a bit confusing.

SJ

paladin




msg:77575
 12:31 am on Feb 19, 2003 (gmt 0)

yes,
mysite.com/_shopByBrand.aspx?brand=Body-Tone

would become

www.site.com/brand/Body-Tone.html which does not exist on the server, calling the 404 page.

The 404 page would then take the "brand/Body-Tone" out of the URL and treat them as if they were "?brand=Body-Tone".

I hope that clears it up.

FYI - I have done similar things in e-commerce sites. What I would usually do is have the catagory page with no querystring and the product pages with the querystring. Something like:
www.mysite.com/catagory-name.asp
www.mysite.com/products.asp?id=55

This made a major difference in the speed that the pages that were crawled and indexed by Google and other SE's. The catagory pages were crawled and indexed almost immediatley, while the product pages were eventually also crawled and indexed, but at least a month (if not more) later.

Something to think about.

Alon

Sammy Jankis




msg:77576
 1:04 am on Feb 19, 2003 (gmt 0)

I just found a company that offers a product to replace query strings with file structure urls. The product is called XQASP filter (http://xde.net/xq/tool.xqasp/qx/index.htm)

Has anyone heard of it?

(Is posting a url against the TOS?)

paladin




msg:77577
 3:22 am on Feb 19, 2003 (gmt 0)

I have heard it referenced a several times. The final effect is the same as changing the 404 file as I mentioned earlier, or manually coding the pages accordingly. The biggest problem that most people would have with it, along with the 404 file idea, is that they have to have their own server, as upposed to virtual server, to install it. For those that do not have access to install programs on their server, or to change the 404 file, they would have to manually code the pages to get this effect. I have done this on several sites.

From what I remember it is against the TOS to post the URL so you may want to edit it. As it is an informational link (at least partially) you may want to ask one of the forum moderators to allow it.

cidrolin




msg:77578
 8:06 pm on Feb 22, 2003 (gmt 0)

I am developing a small php content-management system. I implemented it on a small site that was previously static. At first I generated normal GET-method urls of the kind :
[volavoile.be...]

I soon noticed that GB would no longer index my content. So I found the "slash-separated parameters" solution on this forum, and my urls presently look like :
[volavoile.be...]

Needless to say, the sid is present only if the client refuses cookies, which means that it must be inserted at the entry point (so that the script can check whether the client accepts cookies or not, and not loose the session in any case).

Now, on the day I set up the modified program, I recorded *several hundred* GB hits. GB activity has now come to a more normal rate, but only the sitemap (main entry point) gets indexed...

Any idea?

piffdude




msg:77579
 9:32 am on Feb 27, 2003 (gmt 0)

My limited understanding of the Google bot has lead me to believe that not only is it long dynamic URLs that are not indexed but also if the URL itself is long eg. more than 2 slashes in the URL as well as more than two variables in a '?' delimited URL.

aspdesigner




msg:77580
 10:41 am on Feb 27, 2003 (gmt 0)

cidrolin has a separate post with that same question, see -

[webmasterworld.com...]

GoogleGuy




msg:77581
 3:54 pm on Feb 27, 2003 (gmt 0)

The little mantra I repeat to myself is "dynamic urls ok, avoid sessionids if at all possible." Spiders don't like sessionsids. :)

pgrote




msg:77582
 6:14 pm on Feb 27, 2003 (gmt 0)

Thanks for the clarification. Can you or someone else tell me if this is ok. I have whittled it down to this:

[mysite.com...]

No session ID. No illegal characters. :-)

Grumpus




msg:77583
 6:21 pm on Feb 27, 2003 (gmt 0)

The URL looks fine, but you also have to keep in mind that your PHP code needs to spit out good clean HTML. If it's chock full of errors, the spider won't be able to crawl it.

I assume you understand this, but thought it might bear mentioning.

G.

pgrote




msg:77584
 6:39 pm on Feb 27, 2003 (gmt 0)

I assume you understand this, but thought it might bear mentioning.

Thank you for checking out the URL.

Yes, very good point and one I will look into now.

atadams




msg:77585
 6:45 pm on Feb 27, 2003 (gmt 0)

http://www.mysite.com/modules/one/myfile.php?id=1

Don't use "id=" even if it's not a session ID, there is a very good chance Google won't crawl it. I had several pages that used "id=" on one of my sites and Google wouldn't crawl them for over a year. I changed the "id=" to something else, and Google crawled them the next day.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved