Welcome to WebmasterWorld Guest from 54.226.189.112

Forum Moderators: goodroi

Message Too Old, No Replies

How do I apply the robots.txt?

Seems important and need to know more!

     

askjoe

6:17 pm on Mar 7, 2003 (gmt 0)

10+ Year Member



I know that the robots.txt tells the spider where to go what commands do I use to get it to best do its job? Is this a file that I must attach to my website If I use front page how do I apply it there?

What exactly needs to be done in order to fully maximize my listings with this feature? Your advice is greatly appreciated!

Askjoe

heini

6:23 pm on Mar 7, 2003 (gmt 0)

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member



AskJoe:
[searchengineworld.com...]
That gives you everything that you need.
Basically robots.txt must be in the root folder. It tells bots, what they are allowed to spider, it does not really tell them where to go though.

How to set it up gets explained there very well.
Look under Examples

korkus2000

6:24 pm on Mar 7, 2003 (gmt 0)

WebmasterWorld Senior Member korkus2000 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Try this tutorial
[searchengineworld.com...]

You can also look at these examples

[webmasterworld.com...]
[google.com...]

Make the file and upload it to your root directory. Thats it.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month