homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

robots.txt Disallow query
Disallow specific query in Robots.txt

 10:45 pm on Jan 8, 2009 (gmt 0)


How would I go about Disallowing a query such as:


the numbers of course will change, I would like all "post_id-*.html"
queries Disallowed in robots.txt.

any suggestions?

thnx in advance!



 4:29 am on Jan 9, 2009 (gmt 0)



 12:35 pm on Jan 9, 2009 (gmt 0)

The rule matches "from the left".

More important is to know of similar URLs that you DO NOT wish to disallow.

In your case:

Disallow: /foru

would work, but would obviously disallow a load of other things that you probably do not want to disallow.

You talk about "post_id-*.html" but is there anything you *do* want to allow in the form "post_id-*.<something else>"?


Disallow: /forum/post_id

is all you need.

That will match any URL where the URL begins with /forums/post_id.


 10:35 pm on Jan 9, 2009 (gmt 0)

Thank you for taking the time to reply g1smd!

Actually the post_id should work fine, there is a topic_id as well that lists the exact same content, only threaded, instead of individual posts.

again thank you again for your time. In due time, I am sure I will subscribe to this site, It's very helpful.

regards, frogz

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved