homepage Welcome to WebmasterWorld Guest from 54.234.231.49
register, login, search, subscribe, help, library, PubCon, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library : Charter : Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
robots.txt wildcard syntax?
is this syntax ok?
maha




msg:4320022		 3:49 pm on May 31, 2011 (gmt 0)

I have thousands of these similar url's (see below) that needs to be eliminated from Google index, to avoid duplicate content.

/category6_7/product23/product_info.html
/category2_10/product636/product_info.html
/category77_33/product1171/product_info.html
/category23_35/product705/product_info.html

Is this the correct syntax for robots.txt?

Disallow: /category*/product*/product_info.html


I want to make sure before I make it live.

thanks for your help in advance..

 

g1smd




msg:4320170		 8:00 pm on May 31, 2011 (gmt 0)

It might be.

Disallow: /*/product_info.html
OR
Disallow: /category*/product_info.html

might also be right.
maha




msg:4320185		 8:26 pm on May 31, 2011 (gmt 0)

I'm trying to eliminate duplicate content with these "good" url's:

/product23/product_info.html
/product636/product_info.html
/product1171/product_info.html
/product705/product_info.html

so I want to be very careful with my wildcards.. ;-)
g1smd




msg:4320193		 8:51 pm on May 31, 2011 (gmt 0)

I'd use the last one:

Disallow: /category*/product_info.html
maha




msg:4320218		 9:39 pm on May 31, 2011 (gmt 0)

Thanks for your help..
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
WebmasterWorld ® and PubCon ® are a Registered Trademarks of Pubcon Inc.
© Pubcon Inc. 1996-2012 all rights reserved