homepage Welcome to WebmasterWorld Guest from 54.145.243.51
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
block ? with robots.txt or htaccess
zeus

WebmasterWorld Senior Member zeus us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4462355 posted 1:00 pm on Jun 6, 2012 (gmt 0)

How can I block mysite. ​com​/?i​d=2​891​&ca​t=bla I get a lot of duplicated content of my frontpage, they all starts with ? after the domain

is this the right way in robots txt

User-agent: *
Disallow: /*?

I dont think so, be cause it blocks ? any where right, I just want if it comes after the domain

 

Hoegaarden

5+ Year Member



 
Msg#: 4462355 posted 10:48 pm on Jun 8, 2012 (gmt 0)

You really should be looking into the creation of query string URLs rather than blocking from search engines. Why is your website creating these pages in the first place?
Anyway if you do not want to answer for any reason, in your Goolge Webmaster Tools account you can tell Google Not to index these URL Parameters ?whatever

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4462355 posted 10:52 pm on Jun 8, 2012 (gmt 0)

Parameters right after the domain would be:

Disallow: /?


You should also add

Disallow: /index.php?

or similar.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved