homepage Welcome to WebmasterWorld Guest from 54.205.207.53
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Is this valid? wildcard in robots
Disallow: /*/edit.html
rudyten

5+ Year Member



 
Msg#: 3629963 posted 1:56 pm on Apr 18, 2008 (gmt 0)

User-agent: *
Disallow: /*/edit.html
Disallow: /*/report.html

-------------------------
I have many pages with this format. where the number is an ID
www.example.com/999/report.html

 

johnxarce

5+ Year Member



 
Msg#: 3629963 posted 10:14 am on Apr 20, 2008 (gmt 0)

this is my first i saw robots.txt like this
User-agent: *
Disallow: /*/edit.html
Disallow: /*/report.html

may you can try to use this in normal format

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 3629963 posted 1:47 pm on Apr 20, 2008 (gmt 0)

what you are using is called wildcards or pattern matching. and basically it is telling the search engines not to index and file that has that sequence of character in it. the big search engines support this.

it is not technically valid. the robots.txt protocol does not support wildcards (pattern matching). this means that the other bots will not probably be able to support this.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved