ricardo12
msg:4098643 | 9:24 am on Mar 16, 2010 (gmt 0) |
Hi........ Yes it is ok
|
phranque
msg:4098734 | 1:29 pm on Mar 16, 2010 (gmt 0) |
it depends on what you are looking for.
robots.tx Disallow prevents crawling but it doesn't prevent indexing.
therefore these directives will prevent the content of those pages from being crawled but they won't prevent the urls from being indexed.
that requires a robots.txt that allows crawling and a robots noindex metatag in the document or the appropriate X-Robots-Tag http header.
|
phranque
msg:4098751 | 1:45 pm on Mar 16, 2010 (gmt 0) |
i would also suggest you check out the interview with Matt Cutts by Eric Enge [webmasterworld.com] and discussion on this subject.
|
|
