Welcome to WebmasterWorld Guest from 54.167.216.93

Forum Moderators: goodroi

Message Too Old, No Replies

robots.txt wildcard syntax?

is this syntax ok?

     
3:49 pm on May 31, 2011 (gmt 0)

10+ Year Member



I have thousands of these similar url's (see below) that needs to be eliminated from Google index, to avoid duplicate content.

/category6_7/product23/product_info.html
/category2_10/product636/product_info.html
/category77_33/product1171/product_info.html
/category23_35/product705/product_info.html


Is this the correct syntax for robots.txt?

Disallow: /category*/product*/product_info.html


I want to make sure before I make it live.

thanks for your help in advance..
8:00 pm on May 31, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



It might be.

Disallow: /*/product_info.html

OR
Disallow: /category*/product_info.html


might also be right.
8:26 pm on May 31, 2011 (gmt 0)

10+ Year Member



I'm trying to eliminate duplicate content with these "good" url's:

/product23/product_info.html
/product636/product_info.html
/product1171/product_info.html
/product705/product_info.html

so I want to be very careful with my wildcards.. ;-)
8:51 pm on May 31, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I'd use the last one:

Disallow: /category*/product_info.html
9:39 pm on May 31, 2011 (gmt 0)

10+ Year Member



Thanks for your help..
 

Featured Threads

Hot Threads This Week

Hot Threads This Month