Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: open
I have a database full of different types of products by different manufacturers. Let’s call them widgets. Now on a page on my site you can see the types of widgets we sell. This page contains links to asp pages that carry different widgets by different manufacturers. There are many many widgets in each category, so I have broken the pages up to about 16 widgets per page.
The link to one of these pages looks like this:
Ok, no problem there. In fact, googlebot has gobbled up all of these pages, and every widget found on these pages appears #1 in a google search. Not Bad.
Now, remember, I have broken the widgets up to about 16 widgets per page. So at the bottom of the above mentioned page there is a link that says ‘CLICK HERE TO SEE MORE WIDGETS!’ and the link to the next page will look like this:
the ‘nav=16’ is how I wrote my code so the ASP page will show products 16 through 32. At the end of that page will be a ‘click here to see more widgets’ link that will say:
and so on.
Now, here is my dilemma. Google will NOT spider the pages that contain this ‘nav=’ line in it. Every one of these products from the base pages appears in the google index with a high rank, however anything deeper and googlebot will not spider.
Does googlebot have some sort of flag pop up when it see’s an ‘=(number)’ in a dynamic url? Is there a sneaky way I could get around this? Do I have to rewrite my entire code? If I, say, used letters instead of numbers (nav=BB) and used asp code to translate this into numbers, would googlebot then spider these pages?
We're finalizing this right now on an asp generated database. We've already done it with one and are awaiting final results, we're going to give it 30 days or so before making a final decision. If you can get rid of all the unusual characters, you'll be much further ahead.
Do a site search here for the term "parsing urls" and you should find some valuable information on how to do it for both UNIX and Windows.
currently, to stick the nav in for the asp code to read, I do this:
now, instead, I am going to try a little ASP voodoo. The url will now look like this:
I am going to assume the "~" caracter (what is that called anyways?) is going to be a 'safe' symbol, since I have seen that in web page filenames. none of my products have the '~' symbol in it, so it will not cause mistakes that way. I do not think google will flag a page for this symbol.
now, in the asp page itself, i will seperate 'widgets~15' into two seperate strings by useing a couple of string functions.
nav = right(category, (len(category) - instr(category,"~")))
category = left(category, instr(category,"~")-1)
ta-da! i have seperated that single variable into two seperate variables:
any comments on this? think it will work?
any comments on this? think it will work?
Not too sure but I don't think the tilde is an acceptable replacement. Parsing the URL's is no big deal once you get the hang of it. I dropped you a sticky mail with an example, we're you able to surmise anything from that?
Also keep in mind that there are other SE's besides Google! Many of them cannot get past the question (?) mark. I'd take the plunge now and parse the urls, you'll be thankful you did in the long run.
P.S. Think about your users too. Its much easier to remember this...
It also looks much nicer in the SERP's. ;)
We just installed IISRewrite on the server yesterday. We are experimenting with that to see if it is a viable alternative to customizing each asp database. I'd much rather have the ability to do it at server level instead of for each and every site.
There is no downloadable script that I'm aware of. There are a few software packages that you can install on the server and that is it. The rest of it is custom from the ground up.
Remember.. I am renting hosting space on another company's system.. i have only basic FTP access to modify my site...
I saw the links you sent, and I understand what you are saying, however, in order to be able to parse your urls, do you not require access to the actual configuration of the webserver?
Hmmm, I don't think so with that one because all of the parsing is done through statements at the tops of the main category pages. I'll check with my programmer, he comes in again tomorrow (Wednesday).
When I worked with him on this over the phone, he was accessing the main category and product templates and writing his code in there.
On the search result, it just shows the URL:
On the other hand, the the url like this one got indexed with the correct title.
WEB SITE TITLE
I don't see any difference!
It takes some time to get the pages displayed because they have to parse through dynamic xml data from another site to get the Title.
But the thing is google still indexed the pages, just not with the title information.