Session IDs in URL and spidering

Do URL Session IDs affect spidering?

[Jimmy Turnip]

I've just finished my first site in ASP.NET. My intention was the most amount of visitors possible could use and acccess the site so I opted for the "cookieless = true" option in the config file for the site so that the session ID is passed in the URL. In ASP.NET it seems to pass the ID in a very strange way www.domain.com/(SESSION ID HERE)/page.aspx

My site starts using session variables as soon as a visitor requests the default page and it is important that it does this on the first page.

Does anyone know if this affects how my site will get spidered? I have tried to optimize the site for search engines, but will all this work have gone to waste if I keep the session variables in the URL?

I know you can pass a session variable in the URL with PHP, so maybe you guys can help too?

[korkus2000]

It will most likely effect the spidering if it looks too dynamic or if you cannot hit that URL after the session has ended. I have not personally turned off cookies in the web.config. Post an example url using example.com so I can see what it looks like.

[Jimmy Turnip]

This is what the URL looks like:

[example.com...]

It has no problem accessing the URL if the session has expired. It either assigns the session to that ID, or creates a new one.

What do you think?

[korkus2000]

I think it could hurt you because you will have several pages, to the SE spiders that have the same content. It could trigger a duplicate content penalty. I wouldn't use it.

[TheNige]

I too think that it will hurt....When the spider goes looking to re-index page example.com/23233423/test.asp, it is going to get redirected to example.com/423423442/test.asp

so, your page will be treated like it wasn't there anymore and be removed from the index....and as someone mentioned it could be treated as duplicate content.

[Jimmy Turnip]

I don't think I explained it quite right in my last post actually. After trying it a few times, including after the session has expired, the URL with the session ID can still be hit.

Thing is, how many visitors do you think I am going to restrict from using the site if I use cookies for the session state?

[phaze]

Hi Jimmy,

I had this dilemma a while back when I was creating a large site for the BBC. I ended up rolling my own uri based session management similar to what ASP gives you out the box. And I regretted it. The down sides are security, spidering and the performance overhead of dynamically adding the session to every link in a page, in that order.

Strictly speaking you are giving a spider the perception that you have a bunch of directories on the site that are large Hex numbers.

Your goal (with google anyway) is to gather pagerank for a particular URI. And if all your URI's are different, you're up S**t creek. Also if, for example, google starts crawling pages based on data it gets from the toolbar, you're going to be in trouble.

There's a reason that sites like this one try to avoid putting anything unneccesary in the URI, including a query string. They try to make everything look like a static HTML page even if it is completelly dynamic. It's more spider friendly that way.

Just bite the bullet and use cookies. Most large sites, including google, that have sessions management require cookies.

More importantly, URI based session management is a security risk. Anyone who sees that URI, including a webmaster who see's a referer, can hijack that session. And believe me, I regularly tail my logs watching users as they arrive at the site. If I saw something that looked interesting, I might just visit it for kicks.

There are a few things you can do to make it *slightly* more secure, like killing the session if the referer does not match your site. But many browsers have referer's disabled, so that's not reliable.

Mark.

[Jimmy Turnip]

Thanks for your help.

I'm going to play it safe so I've now removed the Session ID from the url and altered my code so the site asks visitors to enable cookies for the areas that session variables are essesntial. The site looks "neater" without the session id in the address bar too.