Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How to disallow search results

How to disallow search results



7:56 pm on Jul 16, 2008 (gmt 0)

5+ Year Member

I have a website that produces search URLs that look like this:


How do I disallow this?

The site also has a sort function that sorts products on a page. The URL generated is as follows:


how do I disallow robots from crawling these pages?

Please let me know what I should add to the robots.txt page.

I would be really obliged for the help.


8:20 am on Jul 17, 2008 (gmt 0)

10+ Year Member


I had a similar problem. Not sure if this will help but check this thread out



12:29 pm on Jul 17, 2008 (gmt 0)

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

htdawg is referring to using wildcards. wildcards aka pattern matching are supported by google, yahoo and msn. they are not officially part of the robots.txt protocol so many of the smaller bots do not handle them well. be careful with using wildcards with those other bots.


1:28 pm on Jul 17, 2008 (gmt 0)

5+ Year Member

Thanks for the replies..

The sort URL generated looks something like this: product-name?sort1desc=F&sort1=Item_NAME&range=

Would it help if I disallow:

Disallow sort1desc=F&sort1=Item_NAME&range=

I just need to disallow this single page


2:00 pm on Jul 17, 2008 (gmt 0)

10+ Year Member

I think so unless you have urls like this for example

sort1desc=F&sort1=Item_NAME&range=productname3 ect..

cause if you just disallow only sort1desc=F&sort1=Item_NAME&range=

you will still have urls like above I think.

so you might have to try the solution in the link I posted before.

Receptional Andy

2:01 pm on Jul 17, 2008 (gmt 0)

Disallow: /Products?search
will block any url starting with that string (presumably, that would cover all search result pages).

Featured Threads

Hot Threads This Week

Hot Threads This Month