mattg3 - 2:54 pm on Jul 17, 2006 (gmt 0)

If you don't want certain stuff of yours to be indexed, you need to disallow it in your robots.txt file.

So it's there for all the freak stalkers to look at that download the robots.txt .... and go then exactly to these pages ..

What we need is a submitable robots.txt not an open one ...

OK you can play around with access controlling IP adresses and Referrers, it's still no use.

[edited by: mattg3 at 2:57 pm (utc) on July 17, 2006]