Would crawlers crawl this?

Usiing the correct robots.txt technique

[ulysee]

Let's say I have a robots.txt file like this:

User-agent: anybot
Disallow: /redir.php

Would bots be able to follow url's like this:
[domain.com...]
or would it not follow the url above?.

[dmorison]

A robot following robots.txt to the letter would not reach "http://www.anotherdomain.com/" by virtue of requesting redir.php.

However; what the robot might do (and it is believed that Googlebot does this now) is simply add anything that looks like a valid URL to its crawl list; and so "http://www.anotherdomain.com/" would be spotted as a potential crawl target and added to the list.

If you want to make sure that the links are found then don't do what you're proposing; find another way.

[ulysee]

I want to make sure that links are not found by any crawler, any suggestions?.

[dmorison]

Gonna be tricky; you could do something clever with JavaScript; but again, the search engines are starting to parse JavaScript and would be able to uncover anything that you are making available as a click-able link to a human.