Do these robot.txt's look ok?

[Boulder90]

They are for separate URL's. The first is for my hobby wordpress blog:


User-agent: *
# disallow all files in these directories
Disallow: /cgi-bin/
Disallow: /z/j/
Disallow: /z/c/
Disallow: /stats/
Disallow: /dh_
Disallow: /about/
Disallow: /contact/
Disallow: /tag/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /contact
Disallow: /manual
Disallow: /manual/*
Disallow: /phpmanual/
Disallow: /category/

The second is for my commercial static site that also has a forum attached(simple Machine Forum software):

User-agent: *
Disallow:

Disallow: /forums/index.php?action=help
Disallow: /forums/index.php?action=search
Disallow: /forums/index.php?action=login
Disallow: /forums/index.php?action=register
Disallow: /forums/index.php?action=admin
Disallow: /forums/index.php?action=post
Disallow: /forums/index.php?action=who
disallow: /forums/index.php?action=printpage



Thanks for any feedback.

[ricardo12]

Hi........ Yes it is ok

[phranque]

it depends on what you are looking for.
robots.tx Disallow prevents crawling but it doesn't prevent indexing.
therefore these directives will prevent the content of those pages from being crawled but they won't prevent the urls from being indexed.
that requires a robots.txt that allows crawling and a robots noindex metatag in the document or the appropriate X-Robots-Tag http header.

[phranque]

i would also suggest you check out the interview with Matt Cutts by Eric Enge [webmasterworld.com] and discussion on this subject.