homepage Welcome to WebmasterWorld Guest from 54.204.182.118
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Using Wildcat
stormshield

5+ Year Member



 
Msg#: 3686336 posted 2:08 pm on Jun 29, 2008 (gmt 0)

Hi. I have a multilingual site with directories that I wouldn't like to be indexed. URLs at my website look like this:

/en/user/password/
/en/contact

/fr/user/password/
/fr/contact

/de/user/password/
/de/contact

Will that work?

Disallow:/*/user/password/
Disallow:/*/contact

 

ThomasB

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3686336 posted 5:13 pm on Jul 6, 2008 (gmt 0)

This should work for Google, but others, like Yahoo might have problems and it will most probably not work. Reason is, that this is not standardized robots.txt code.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved