This permits all robots, right? User-agent: * Disallow:
Are there any more syntax I have to add to my sites head like: <META NAME="robots" CONTENT="index"> is this required when I already have a robots.txt?
whats the best settings, index:follow?
benevolent001
4:08 am on May 5, 2005 (gmt 0)
Hi
Yes robots.txt is right
For mor info on meta tags you may just google for meta tags the first result page gives good info and there are other meta tags too which are important like desc and keywords etc...just use google
Lord Majestic
8:16 pm on May 5, 2005 (gmt 0)
<META NAME="robots" CONTENT="index"> is this required when I already have a robots.txt?
This is not necessary as assumed defaults are index and follow. Having empty or allow-everything robots.txt is only good to avoid 404 errors in log files.