Welcome to WebmasterWorld Guest from 54.159.50.111

Forum Moderators: goodroi

Message Too Old, No Replies

How about these code

     
11:00 pm on Mar 28, 2010 (gmt 0)

New User

10+ Year Member

joined:Jan 16, 2004
posts:34
votes: 0


I do not want search engines to craw the certain cotents. How about the following code?

User-agent: *
Disallow: /cheapprescription/
Disallow: /cheap-glasses/?Cheap-Glasses-
Disallow: /cheap-glasses/?discount-glasses-
Disallow: /cheap-glasses/?Men-Glasses-
Disallow: /cheap-glasses/?Women-Glasses-
Disallow: /cheap-glasses/?Kids-Glasses-
Disallow: /cheap-glasses/?Eyeglasses-Glasses-
Disallow: /cheap-glasses/?Men's-Glasses-
Disallow: /cheap-glasses/?Women's-Glasses-
Disallow: /cheap-glasses/?kids-Glasses-
Disallow: /cheap-glasses/?Fashion-glasses-
Disallow: /cheap-glasses/?Prescription-Eyeglasses-
Disallow: /cheap-glasses/?Bifocal-Glasses-
Disallow: /cheap-glasses/?Reading-Glasses-
Disallow: /cheap-glasses/?Plastic-Eyeglasses-
Disallow: /cheap-glasses/?Prescription-glasses-
Disallow: /cheap-glasses/?Classic-glasses-
Disallow: /cheap-glasses/?Pure-Titanium-Eyeglasses-
Disallow: /cheap-glasses/?Semi-Rimless-Eyeglasses-
Disallow: /cheap-glasses/?Rimless-Glasses-
Disallow: /cheap-glasses/?Full-Rim-Glasses-
Disallow: /cheap-glasses/?Bendable-Titanium-Eyeglasses-
Disallow: /cheap-glasses/?Progressive-Glasses-
Disallow: /cheap-glasses/?eyeglasses-Progressives-
Disallow: /cheap-glasses/?eyeglasses-Bifocals-
Disallow: /cheap-glasses/?Full-Rim-Eyeglasses-
Disallow: /cheap-glasses/?Rimless-Eyeglasses-
Disallow: /cheap-glasses/?size0-
Disallow: /cheap-glasses/?size1-
Disallow: /cheap-glasses/?size2-
Disallow: /cheap-glasses/?size3-
Disallow: /cheap-glasses/?size4-
Disallow: /cheap-glasses/?price0-
Disallow: /cheap-glasses/?price1-
Disallow: /cheap-glasses/?price2-
Disallow: /cheap-glasses/?price3-
Disallow: /cheap-glasses/?Full-Rim-Sunglasses-
Disallow: /cheap-glasses/?Rimless-Sunglasses-
Disallow: /cheap-glasses/?Semi-Rimless-Sunglasses-
Disallow: /cheap-glasses/?Pure-Titanium-Sunglasses-
Disallow: /cheap-glasses/?Metal-Sunglasses-
Disallow: /cheap-glasses/?Bendable-Titanium-Sunglasses-
Disallow: /cheap-glasses/?Kids-Plastic-Glasses-
Disallow: /cheap-glasses/?Kids-Metal-Glasses-
Disallow: /cheap-glasses/?Kids-Full-Rim-Glasses-
Disallow: /cheap-glasses/?Kids-Bendable-Titanium-Glasses-
12:09 am on Apr 2, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


This would remove more than half of the above code:

Disallow: /cheap*glasses
Disallow: /cheap*Glasses