How to stop Googlebot from following some links

newbie question re robots.txt

[TylerDurden]

Sorry if my question sounds obvious.
I have a new site and Googlebot is currently crawling it first time. I have robots.txt file where I disallowed some files from being spidered. In particular I would like to disallow Google to follow ALL affiliate links on my site. All those links are going through php redirect script with parameters (goto.php?par=XX). I've put the following into my robots.txt file:
User-agent: *
Disallow: goto.php
However now I can see that Google still following those links. I've read through different robots.txt documentation and haven't found how to properly use wildcards in URL in robots.txt.
Should I say "Disallow: goto.php?*" or "Disallow: goto.*" or something else.
Sorry again if my question is stupid but I'm paranoid about made something wrong in robots.txt and erroneously stops my site from spidering (especially by Google).
Tyler

[fathom]

You can disallow googlebot from specific directories, and pages but I don't believe you can instruct it not to index/crawl, and/or follow specific "links", it's the complete page (all links) or nothing.

Use Javascript links on any link not to be followed.

[tedster]

There's some good Robots.txt information [searchengineworld.com] on our sister site, SearchEngineWorld.com

Disallow
The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that starts with this value will not be retrieved.

For example, Disallow: /help disallows both /help.html and /help/index.html, whereas Disallow: /help/ would disallow /help/index.html but allow /help.html.

You don't need to use a wildcard character. As the above snippet says, "Any URL that starts with this value will not be retrieved." So, you're on the right track, but you do need the slash! Try:

disallow: /goto

...assuming that no links except your affiliate links begin with "/goto"

[fathom]

Learn something here everyday. ;)

[TylerDurden]

Thanks a lot Tedster!
Is it OK to change robots.txt in the middle of crawl?
I've noticed that Googlebot usually (but not always!) re-asking robots.txt before every new visit. May the change cause a confusion?

[aspdesigner]

Have you considered maybe putting this file in a directory, and then blocking the entire directory from being spidered?

You can also use NOINDEX to prevent Google from including this file.

[edited by: aspdesigner at 7:51 pm (utc) on Jan. 5, 2003]

[bobmark]

You can also use the "noindex" and/or "nofollow" metatag on specific pages. I find this useful for sub-pages that I have no desire to have indexed, but do not want to add to .htacess or robots.txt as they may be temporary in nature. And it is always good to keep generalized files, especially .htaccess, as lean as possible, both for access times and just because you end up with this huge indecipherable block of code that's hard to edit if you use it to control access to a ton of individual pages on a large site.