Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: goodroi
I know virtually nothing about robots.txt so forgive me if this question is silly. I figured better safe than sorry. Anyway...
For whatever reason this site has it's directory structure setup like this:
They want to block only the /company.php/* ones for fear of overlapping duplicate content (many of the companies only have only 1 product and thus the pages are virtually identical).
I was just wondering what the correct disallow syntax is for that? Thanks!