I am obviously very new to web building. I have read lots of places that one must create a robots.txt file in order for search engines to visit. Is this something that needs to be in my html or is this a file that the host server must set up?
treeline
4:09 am on Aug 28, 2005 (gmt 0)
This is a simple text file that you setup. It is not necessary if you want search engines to view your whole site. It is used to block search engines from viewing certain areas, or if you get tired of seeing 'file not found' in your log files when robots look for it.
