Forum Moderators: open
2003-11-22 19:04:11 64.68.84.143 - GET /ProductCart/pc/viewCat_P.asp 200 0 HTTP/1.0 Googlebot/2.1+(+http://www.googlebot.com/bot.html)
Now normally the file structure would be like this:
/Productcart/pc/viewCat_P.asp?idCategory=2
Google seems to be crawling my site over and over. But when looking at the raw logs all I see it looking at is "/Productcart/pc/viewCat_P.asp".
Not "/Productcart/pc/viewCat_P.asp?idCategory=2" like I see for other visitors.
I've heard google will index dynamic pages, I have heard it will not index them when they are "asp?id=2", but will it not index or look at it when it's something like "asp.?idCategory=2"
Over the weekend I made some dramatic changes based on some suggestions I received.
I made it so that only the index page, policies pages, dynamic product pages and dynamic category pages have robot=index.follow on them. all others have robots=noindex, nofollow.
I built an extensive site map, feeding to category pages so that I could stay under the 100 links recommendation. of course these should be deadends at the product pages if google observes the noindex,nofollow meta tags
I included an empty robots.txt file, just for the sake of have a robots.txt
My site is hosted on an IIS 5.0 machine, my host won't install url rewrite software (windows version of apache mod rewrite).
So the question is, what now? Help I'm going to blow away with no good index by the SE's
It is very easy to turn dynamic urls into 'nice' ones, so don't go the hard way.
Take a look at mod_rewrite and rewrite your urls.
My site is hosted on an IIS 5.0 machine, my host won't install url rewrite software (windows version of apache mod rewrite).
It may be time to locate a new host. I see many who make the same comments. If web hosting companies are not flexible enough to install URI Rewriting software, then they do not need your business.
