Welcome to WebmasterWorld Guest from 54.227.110.209

Forum Moderators: goodroi

Message Too Old, No Replies

robots.txt wildcard syntax?

is this syntax ok?

     

maha

3:49 pm on May 31, 2011 (gmt 0)

10+ Year Member



I have thousands of these similar url's (see below) that needs to be eliminated from Google index, to avoid duplicate content.

/category6_7/product23/product_info.html
/category2_10/product636/product_info.html
/category77_33/product1171/product_info.html
/category23_35/product705/product_info.html


Is this the correct syntax for robots.txt?

Disallow: /category*/product*/product_info.html


I want to make sure before I make it live.

thanks for your help in advance..

g1smd

8:00 pm on May 31, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



It might be.

Disallow: /*/product_info.html

OR
Disallow: /category*/product_info.html


might also be right.

maha

8:26 pm on May 31, 2011 (gmt 0)

10+ Year Member



I'm trying to eliminate duplicate content with these "good" url's:

/product23/product_info.html
/product636/product_info.html
/product1171/product_info.html
/product705/product_info.html

so I want to be very careful with my wildcards.. ;-)

g1smd

8:51 pm on May 31, 2011 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I'd use the last one:

Disallow: /category*/product_info.html

maha

9:39 pm on May 31, 2011 (gmt 0)

10+ Year Member



Thanks for your help..
 

Featured Threads

Hot Threads This Week

Hot Threads This Month