Are URLS with a '?' spiderable?

Making a dynamic site spiderable.

[crazed canuck]

It is my understanding that dynamic sites that have a query string appended to the url are difficult for a search engine to spider. A spider obviously wouldn't fill out a form to get the results of a dynamic page to index but do spiders generally have a problem with any '?' in the url.

My reason for asking is that I want to make a site map that the search engine spiders can follow. I want to 'hard code' the search parameter into links so that page (hopefully) comes up in the search engine resultes pages (SERPS).

For example, I wish to have a query of widgets in a city come up which would normally be done through an internal search form but on the site map I would have a hard link:

<a href="http://someportal.com?type=widgets&city=somecity">Find City Widgets</a>

Basically, I am hoping that a 'filled in' link will be spiderable. Any comments or other suggestions in making a dynamic site spiderable?

[edited by: agerhart at 5:45 pm (utc) on Sep. 24, 2003]
[edit reason] removed specifics [/edit]

[closed]

As long as links to your dynamic pages appear on static pages, you should be fine. The spiders should be able to follow the links.

[nakulgoyal]

Do you have? type query pages? If yes, I have a suggestion.

[crazed canuck]

Do you have? type query pages? If yes, I have a suggestion.

Yes, and any suggestions would be most welcome!

[killroy]

Appearantly most spiders will follow links with a? as long as it does not have too many parameters (more then 2 seems a good rule of thumb). Additionally there are some stop parameters like SID or ID which seem to stop spiders as they usually indicate Session IDs.

I've had no troubles with links of the kind you indicated.

SN

[nakulgoyal]

THIS IS NOT MY OWN POST BUT SOMETHING I ARCHIVED SOME TIME BACK. IF YOU KNOW TEH AUTHOR, PM ME OR HAVE HIM PM ME SO THAT I COULD REFER HIM AS THE PERSON WHO GOT ME THIS INFO.
----------------------------------------------------------
Dynamic URLS --> Static Representation
I'm using a simple CMS that references content with?id=27. I know that Google will spider some dynamic links but to be on the safe side I want to try and make that static.

I have created an .htaccess file with:

<FilesMatch "^id$">
ForceType application/x-httpd-php
</FilesMatch>

And then another file "id" with:

<?php

$idpath = explode("/",$_SERVER['PATH_INFO']);
$newid = intval($idpath[1]);

include("http://www.mysite.com/?id=$newid");

?>

What this does it turn:
[mysite.com...]
into:
[mysite.com...]

Two dilemmas:

I don't want to have to use an absolute path in the id file because this feature is going to be made available to other users of the CMS I'm using.

When you use the static path, relative links in the html that is included are broken. This is because they think they are in another subdirectory when in fact it's just smoke and mirrors.

So instead of rewriting all my html with absolute paths or doing some other hack, I would like to accomplish this:

[mysite.com...]

That way the relative links work okay. I was looking at Apache core directives and came up with something like:

<Location ~ "/id*">
//some type of parsing here
</Location>

But I'm rather new to both php and .htaccess so for now this is getting the best of me. I know what I want to do logically but don't know what to do syntactically.

[surfgatinho]

YES, YES and YES - up to a point!
The first is definitely in Google additional query string values are a little vaguer