| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, login, search, subscribe, help, library, PubCon, announcements, recent posts, open posts,
|Subscribe to WebmasterWorld|
|Block Rule: Robots.txt - Will Sub Dir also get blocked?|
| 7:24 pm on Nov 2, 2010 (gmt 0)|
A CMS system we use has an enquiry form that when clicked grays out the page...
When scanning the site, it reports it as a 404, and there are 100's..
The url is as follows www.domain.com/product/enquiry_form/product-abc
There are URL's that come after /product/ that I dont want to block. Would I just need to block /enquiry_form/ and not include product?
| 10:26 pm on Nov 2, 2010 (gmt 0)|
The first product in the URL a literal or a variable?
will stop well behaved bots from requesting.
It won't block
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld ® and PubCon ® are a Registered Trademarks of Pubcon Inc.
© Pubcon Inc. 1996-2012 all rights reserved