Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: goodroi
and on every page of my site I have the meta tag
<META NAME="robots" CONTENT="index,follow">
is this right?
In my root directory, there is one index.html file which is our site, but at the beginning of building it, this was a splash page (not any more) so I had the first page as index2.htm. I did not want to submit the splash page to search engines and such so I submitted the index2.htm page (which is indexed) and so now I have that page in my root directory also, along with one more page and some js files.
My main site is in a directory called page. Do I have this set right for the search engine or robots to crawl this site? Or do I need to change something?
[edited by: DaveAtIFG at 6:11 pm (utc) on April 5, 2004]
[edit reason] Url removed [/edit]
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
If you want to exclude crawlers modify your robots.txt file, if you don't have one create one in notepad, save it as robots.txt then ftp it up to your site, example below;