Welcome to WebmasterWorld Guest from 18.104.22.168 , register , free tools , login , search , subscribe , help , library , announcements , recent posts , open posts Subscribe to WebmasterWorld
Robot.txt - What is it, and why use it laffe msg:1525736 11:45 pm on Oct 30, 2003 (gmt 0) I know it is about protecting misc folders, but what if I dont have one at all? Am I missing something?
Is this a must?
I also read something about cloaking it?
Can someone please show a file? Made the best way?
moltar msg:1525737 11:46 pm on Oct 30, 2003 (gmt 0)
Here you can find a perfect tutorial on robots.txt:
...] searchengineworld.com heini msg:1525738 11:52 pm on Oct 30, 2003 (gmt 0)
Also we have now a brand new shiny forum dedicated to robots.txt, just take a stroll and you will find lots of introductory threads. As to cloaking your robots.txt, here's the discussion: [ ...] webmasterworld.com
And welcome to WebmasterWorld, Leif-Harald.
laffe msg:1525739 4:36 am on Nov 2, 2003 (gmt 0)
thanks people. Helped a lot