homepage Welcome to WebmasterWorld Guest from 54.204.142.143
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Is it possible to block file names beginning with a text string?
Gemini23




msg:4622337
 1:07 pm on Nov 10, 2013 (gmt 0)

In Robots.txt is it possible to block more than 1 file name beginning with the same text string?

wordpress creates a number of urls that I don't want indexed - such as the search pages...

/blog/s?=****
?attachment****

The * represent various different and unique character strings.

Is it possible to block all beginning with >
/blog/s?=
and
/?attachment

 

phranque




msg:4622461
 8:07 am on Nov 11, 2013 (gmt 0)

that's how it works in robots.txt - disallowed urls are matched left-to-right.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved