Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Correctly disallowing only parts of a folder


echwa - 9:50 pm on Aug 16, 2009 (gmt 0)


You could always config your webserver to process # symbol as a ? and this way crawlers will treat your variations:

.com/widget-something/#category=something1&page=&text=0
.com/widget-something/#category=something2&page=&text=0

as the single entity of:

.com/widget-something/


Thread source:: http://www.webmasterworld.com/robots_txt/3971082.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com