homepage Welcome to WebmasterWorld Guest from 54.243.17.133
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
block ? with robots.txt or htaccess
zeus




msg:4462029
 1:00 pm on Jun 6, 2012 (gmt 0)

How can I block mysite. ​com​/?i​d=2​891​&ca​t=bla I get a lot of duplicated content of my frontpage, they all starts with ? after the domain

is this the right way in robots txt

User-agent: *
Disallow: /*?

I dont think so, be cause it blocks ? any where right, I just want if it comes after the domain

 

Hoegaarden




msg:4463163
 10:48 pm on Jun 8, 2012 (gmt 0)

You really should be looking into the creation of query string URLs rather than blocking from search engines. Why is your website creating these pages in the first place?
Anyway if you do not want to answer for any reason, in your Goolge Webmaster Tools account you can tell Google Not to index these URL Parameters ?whatever

g1smd




msg:4463164
 10:52 pm on Jun 8, 2012 (gmt 0)

Parameters right after the domain would be:

Disallow: /?


You should also add

Disallow: /index.php?

or similar.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved