Forum Moderators: open
In early October 04 our website finally went live. We paid a Web Design and SEO company to construct, Host and maintain the site. All was going great until I noticed that virtually none of our Product pages have every been crawled and none of them have PR. Our site is ASP and around 800 pages in size, after doing a bit of homework I noticed that all of our URL's have the? symbol appearing in them and every URL which has this symbol has not been Crawled. At the end of the day we have been left with a homepage which is now PR6 and 12 pages spidered out of a possible 800!
I have spoken to the company and they have come back to me with "this is an industry standard" and "You should of mentioned this initially when you listed your requirements". The questions I have are:
1. Is this something they should really of known?
2. Is this something that can be easily fixed?
3. what would you guys recommend.
They have come back to me with the following solution.
1. Reap the site to create a "flat" version and copy this to the root file on the server. This will create 2 sites identical - one being .asp and the other .htm. (this seems a bit strange to me as surely this will be seen as duplicate content?
I would really appreciate your views on this as I am not very clued up on the Dev side of things (youve probably noticed!) and I need to get this sorted ASAP
This transform www.example.com/page.aspx?a=1&b=2 into www.example.com/page.aspx/a/1/b/2.
If you have a lot of url variables then it means that the page appears to be in a deep directory, but it's better than nothing.
Also create flat files in the root, and link to them from the homepage.
[edited by: Xoc at 12:02 am (utc) on Feb. 10, 2005]
[edit reason] changed to use example.com [/edit]
I've got a few sites where the pages are product.asp?a=4&b=5 type that are indexed by google just fine.
In the past people said don't have more than 3 variables, but I think recently google can handle more.
One problem could be that one of your variables is "id". Google doesn't seem to like them, so if that can be changed it could help.
Personally I wouldn't go for the htm and asp versions as that might end up with duplicate content issues, and sounds like additional work maintaining the pages etc.
If you can can cut down the number of variables your product pages have, get rid of any mention of "id" (including "prodid", "catID", etc) and make sure the categories, and product pages are easily spiderable - have you got a site map linked from the home page, could you have a "featured" product deep linking from the homepage (rotated daily)? - it should be good....
[added]ah - sanenet posted whilst i was typing - that's another top suggestion :)[/added]
[edited by: mcavill at 12:46 pm (utc) on Feb. 1, 2005]
Suppose your site is www.mysite.com
Then for example - what are the urls of a few pages?
Is is like this -
www.example.com/?1
www.example.com/?2
www.example.com/?3
or
Is it like this -
www.example.com/index.html?Page=1
www.example.com/index.html?Page=2
www.example.com/index.html?Page=3
AND
are the keywords & <title>Pagetitle </title> tag values same on all pages / are they different.
[edited by: Xoc at 12:03 am (utc) on Feb. 10, 2005]
[edit reason] changed to use example.com [/edit]
How does it help? (Sorry, i am new in this thing).
Regards,
MB
look at the home page here - deep links into the stuff that matters :)
If you use a form to submit a query and that's the only way to get to the dynamic pages, then spiders probably won't find them. Like others here, I have webmastered sites with thousands of asp pages with the?something=something in the URL and they're in the index of the major engines.
Since development has already cost you, and these developers are in some ways correct about a requirements shift, maybe just ask them to make sure "id" does not appear anywhere in the querystring as a variable name, and make sure that you can hit all 800+ pages by clicking links, not using a form submission. It also takes a while for spiders to deep-crawl, so you may just be expecting too much, too soon :-)
Best wishes
Sanenet - when you mentioned create flat files in the root and link to them from the homepage, do you basically mean create a sitemap? or reap the site and add it to the route, I was worried about doing this due to duplicate content?
what do you all think?
When I said "flat files in the root", I meant your top product pages, information about the company, and specific pages targetted towards your top products. Basically, targetted content rich pages that the SEs will pick up on.
