homepage Welcome to WebmasterWorld Guest from 23.20.77.156
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
Forum Library, Charter, Moderators: phranque

Website Technology Issues Forum

    
Millions of duplicate pages, JSESSIONID
Are robots seeing ;JSESSIONID=#*$! as unique URLs?
brycen




msg:3176535
 8:44 am on Dec 3, 2006 (gmt 0)

I'm getting bit by this issue:
http#//example.com/post/javas-seo-blunder-jsessionid

Google, Alexa and Exalead are each crawling each and every page of my site dozens of times each day. Good you say? Bad, because they're getting pages that differ ONLY in the URL:

93.47.80.51 - - [28/Nov/2006:16:05:36 -0800] "GET
/awards.do;jsessionid=68B86DFF8E4A8597B210531C3431965D HTTP/1.1" 200
17195 "-" "Exabot/3.0"
193.47.80.51 - - [28/Nov/2006:16:17:30 -0800] "GET
/awards.do;jsessionid=0621414681C92E1A00A9428A7800AC30 HTTP/1.1" 200
17195 "-" "Exabot/3.0"
193.47.80.51 - - [28/Nov/2006:17:00:36 -0800] "GET
/awards.do;jsessionid=0079FCD91ED8E5B86902228D285CCEEF HTTP/1.1" 200
17195 "-" "Exabot/3.0"
193.47.80.51 - - [28/Nov/2006:20:41:50 -0800] "GET
/awards.do;jsessionid=DE9B61384D3D75DE9EB38A21F066E433 HTTP/1.1" 200
17195 "-" "Exabot/3.0"

So folks: watch out for this. You may have to move away from Tomcat if you want to solve this issue.

[edited by: txbakers at 11:39 am (utc) on Dec. 4, 2006]
[edit reason] examplified URL [/edit]

 

phranque




msg:3176665
 1:39 pm on Dec 3, 2006 (gmt 0)

not sure what tomcat is but you must disable sessions for bots - especially the search spiders.

txbakers




msg:3177422
 11:40 am on Dec 4, 2006 (gmt 0)

Tomcat is an open source JSP application server

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved