Robots.txt Newbie Question

[paulpsd]

I've been a web developer for years, but am pretty new to the concept of robots.txt. Perhaps somebody can point me to a general thread that explains it, or just answer my simple questions.

As I understand it, robots.txt limits the indexing of your site for whichever searchbots or spambots you mention. Does it also facilitate indexing in some way? If I do not put a robots.txt in my site at all, will my page ranking with some search engines suffer?

I've found a way around email harvesting robots, by using a javascript function to put together an email address. Given that, do I really need any further protection for indexing spiders?

[topr8]

robots.txt only prevents spidering it doesn't enable spidering.

unfortunately there are many bad bots that ignore it, so it doesn't prevent many site rippers and scrapers anyway.

>>If I do not put a robots.txt in my site at all, will my page ranking with some search engines suffer?

imo not at all

[paulpsd]

Thanks! What I gleaned from your reply is that I don't need to worry myself about this topic. Nice. One less thing.

[zeus]

Im introuble with hijackers- redireting sites, can this help, I have never used robot text

User-agent: *
Disallow: /tracker2.php?url=http://www.mydomain.com%2Fmykeyword.2.htm
Disallow: /tracker2.php?url=http://www.mydomain.com%2Fpocket-pc.htm
Disallow: /tracker2.php