Welcome to WebmasterWorld Guest from 188.8.131.52 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Pubcon Platinum Sponsor 2014
How to block this type of URL's iCyborg
well I want to block the url's
but want to keep these url's
so basically I just want to block all pages which are ending with "?theme=xyz" as this is causing unnecessary content duplication.
I want to prevent them from being indexed netchicken1
Shouldn't there be some * too ? firefoxin
How to block this type of url in robot.txt
Which URLs are similar to those, but do not need to be blocked?
Otherwise is this what you want?
Be aware that Google will still list "blocked" URLs as URL-only entries in the SERPs.
/comments.php?id=3740&replyid=30574&catid=1 [remove] /comments.php?shownews=3740 [OK]
i want to remove first two line and 3th one is my primary link.
i think i must use this code :
Disallow: /*id Disallow: /*replyid
is that true ?
Your last suggestion is probably too wide. It will block anything with "id" or "replyid" anywhere in the URL or in the parameters. That's likely going to block some stuff that you don't want blocked. That is, "id" would block anything with "catid" and "ocid" and "docid" as well.
Do you know if the parameters ever appear in a different order?
Do you know if URLs with
shownews ever have additional parameters and that you will not want to block those?
Otherwise, I would do:
User-agent: * Disallow: /comments.php?id= Disallow: /comments.php?action= Disallow: /comments.php?highlight= Disallow: /comments.php?ocid= Disallow: /comments.php?replyid= Disallow: /comments.php?catid=
Maybe others too?
You need to be aware of every possible format that could be requested.
You also need to be aware that with parameters in a different order you have a duplicate URL for the same content.
Yes, the last suggestion is too wide (by far) and doubly redundant. :) Referring to: Disallow: /*id Disallow: /*replyid the case of "/*id" will disallow /avoidupois, /ridiculous, /rancid, /recidivism, /riddle, /typhoid, /zircofluoride, etc., usw., and anything that looks like: "/*replyid".