Welcome to WebmasterWorld Guest from 220.127.116.11
Forum Moderators: goodroi
I think you've misunderstood the role of the robots.txt file - it's not to define keywords, but to define which parts of your site can and cannot be spidered by search engine robots.
The best place to start is the following robots.txt tutorial, complete with examples:
ive added at the very top:
& ive uploaded it, when I view it from my website
it isnt at the top where I saved it, any ideas?
so is there no point putting any keywords in it at all,
so how does robots.txt work?
and shall I put all keywords into to meta? will search engines crawl to my site?
- No, it is just a set of instructions for the bots to follow.
<< ive added at the very top:
it isnt at the top where I saved it, any ideas?>>
- not entirely sure what you mean by this, but to clarify, the robots.txt
Should be in a text file in your website directory called robots.txt. Not written on the actual web page.
If you knew this already, then check your host allows it, I can’t imagine that they would not allow it, but I have been caught with my pants down for assuming things before.
Please clarify what you mean by “ive added at the very top” and ill try to answer.
<< and shall I put all keywords into to meta? >>
- Yes, this is becoming redundant on some search engines, Google for example. But, even if meta tags are not as important as they once were, they are still used and every little helps. Read more about meta tags because you can use them to help give good descriptions of your site when displayed in search results. Also you never know when they will make a comeback.
<< will search engines crawl to my site? >>
- 1, if there are inbound links to your site (links from other websites to yours) then search engines will follow those links and crawl your site.
2, you can ensure engines crawl your site by adding your URL (www.domainname.com) to the search engines. There is normally a link for this on the search engines web site, or you can search add site or something to that affect.
Read, read, read… this is a great place to get all this information, if I were you I would just read every thing you can on webmasterworld.
p.s hello all, great forum, definatly worth signing up for the paid membership just our of appreciation!