| How about these code
|
abiao
msg:4106159 | 11:00 pm on Mar 28, 2010 (gmt 0) |
I do not want search engines to craw the certain cotents. How about the following code?
User-agent: *
Disallow: /cheapprescription/
Disallow: /cheap-glasses/?Cheap-Glasses-
Disallow: /cheap-glasses/?discount-glasses-
Disallow: /cheap-glasses/?Men-Glasses-
Disallow: /cheap-glasses/?Women-Glasses-
Disallow: /cheap-glasses/?Kids-Glasses-
Disallow: /cheap-glasses/?Eyeglasses-Glasses-
Disallow: /cheap-glasses/?Men's-Glasses-
Disallow: /cheap-glasses/?Women's-Glasses-
Disallow: /cheap-glasses/?kids-Glasses-
Disallow: /cheap-glasses/?Fashion-glasses-
Disallow: /cheap-glasses/?Prescription-Eyeglasses-
Disallow: /cheap-glasses/?Bifocal-Glasses-
Disallow: /cheap-glasses/?Reading-Glasses-
Disallow: /cheap-glasses/?Plastic-Eyeglasses-
Disallow: /cheap-glasses/?Prescription-glasses-
Disallow: /cheap-glasses/?Classic-glasses-
Disallow: /cheap-glasses/?Pure-Titanium-Eyeglasses-
Disallow: /cheap-glasses/?Semi-Rimless-Eyeglasses-
Disallow: /cheap-glasses/?Rimless-Glasses-
Disallow: /cheap-glasses/?Full-Rim-Glasses-
Disallow: /cheap-glasses/?Bendable-Titanium-Eyeglasses-
Disallow: /cheap-glasses/?Progressive-Glasses-
Disallow: /cheap-glasses/?eyeglasses-Progressives-
Disallow: /cheap-glasses/?eyeglasses-Bifocals-
Disallow: /cheap-glasses/?Full-Rim-Eyeglasses-
Disallow: /cheap-glasses/?Rimless-Eyeglasses-
Disallow: /cheap-glasses/?size0-
Disallow: /cheap-glasses/?size1-
Disallow: /cheap-glasses/?size2-
Disallow: /cheap-glasses/?size3-
Disallow: /cheap-glasses/?size4-
Disallow: /cheap-glasses/?price0-
Disallow: /cheap-glasses/?price1-
Disallow: /cheap-glasses/?price2-
Disallow: /cheap-glasses/?price3-
Disallow: /cheap-glasses/?Full-Rim-Sunglasses-
Disallow: /cheap-glasses/?Rimless-Sunglasses-
Disallow: /cheap-glasses/?Semi-Rimless-Sunglasses-
Disallow: /cheap-glasses/?Pure-Titanium-Sunglasses-
Disallow: /cheap-glasses/?Metal-Sunglasses-
Disallow: /cheap-glasses/?Bendable-Titanium-Sunglasses-
Disallow: /cheap-glasses/?Kids-Plastic-Glasses-
Disallow: /cheap-glasses/?Kids-Metal-Glasses-
Disallow: /cheap-glasses/?Kids-Full-Rim-Glasses-
Disallow: /cheap-glasses/?Kids-Bendable-Titanium-Glasses-
|
g1smd
msg:4108639 | 12:09 am on Apr 2, 2010 (gmt 0) |
This would remove more than half of the above code:
Disallow: /cheap*glasses
Disallow: /cheap*Glasses
|
|
|
