Welcome to WebmasterWorld Guest from 54.163.25.166

Forum Moderators: goodroi

Message Too Old, No Replies

Robot.txt: Wildcards

     
6:49 pm on Jul 27, 2009 (gmt 0)

New User

5+ Year Member

joined:Aug 29, 2008
posts:14
votes: 0


Question for the group:

If I have a url with the following string:
[mysite.com...]

Let's say I want to disallow everything after the query value. Would the following robot.txt disallow be applicable:
User-agent: *
Disallow: /sub1/*/*/*.html?

?

Any help is much appreciated

12:54 pm on July 28, 2009 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3080
votes: 67


For your robots.txt to block access to all URLs that include a question mark, you can add this entry:

User-agent: *
Disallow: /*?

This is not part of the official robots.txt protocol but it is supported by Google, Yahoo and Bing.