Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How should I use a robots.txt file on a virtual account?

I want to ban spiders from one site only

9:08 pm on Jun 9, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 13, 2002
votes: 0

I have a virtual hosting account on the same IP address and have 7 sites on it. Each site has its own folder with its own index.html file in it etc. I currently have no robots.txt files at all.

I want to ban all spiders from one of my sites, but have the other 6 continued to be crawled normally. How should I go about doing this? (I am scared of accidentally banning spiders from all 7 sites).

My guess is to put the following robots.txt file in the same directory where my index.html is for the site I want ignored:

User-agent: *
Disallow: /

Is that correct? Or would a robots.txt file have to go in the root dir of my virtual account?

Any help would be greatly appreciated.

9:41 pm on June 9, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
votes: 0

Don't think server files and directories, think URLs.

Put it wherever http://www.ignoredsite.com/robots.txt will access it. And yes, that's probably the same directory as the index file, but it does depend on your server configuration. But just upload it there and test by typing the robots.txt URL into your browser.

If you're worried about the other domains, then test them, too. If you haven't put a robots.txt on them, you should get a 404.


2:01 pm on June 10, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 13, 2002
votes: 0

Thanks for the tip Jim, I tested it how you suggested and the robots.txt appeared for the ignored-site.com and I got 404 for the others, so it appears to have worked. :)

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members