homepage Welcome to WebmasterWorld Guest from 54.204.58.87
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

  posting off  
deny Scooter IP.
bignooz




msg:16611
 1:50 pm on Apr 2, 2003 (gmt 0)

Hi there, I try to stop Scooter request from IP 216.39.48.X
Whit no results :(
I have put robots.txt file in /root/ of the server and also in local for each domain server :

User-agent: Scooter
Disallow: /

May be I have done something wrong? Please help is welcome

 

jdMorgan




msg:16612
 2:02 pm on Apr 2, 2003 (gmt 0)

bignooz,

Put your robots.txt in the root directory of each site - in the same directory as your main "index.html" page. Robots will look for it there, and nowhere else. Then, validate your robots.txt files here [searchengineworld.com].

HTH,
Jim

bignooz




msg:16613
 2:19 pm on Apr 2, 2003 (gmt 0)

Thank you Jim!
One thing, put the robots.txt in each domain root is not effective as i am using .htaccess file with "rewrite" index.php orders.
Put the robots.txt file just in the serverīs root can help?

jdMorgan




msg:16614
 2:40 pm on Apr 2, 2003 (gmt 0)

No, It must be (or appear to be, by using an internal redirect rewrite) at [yourdomain.com...]

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved