homepage Welcome to WebmasterWorld Guest from 23.22.76.170
register, login, search, subscribe, help, library, PubCon, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library : Charter : Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Yahoo and Robots.txt
Brett_Tabke




msg:1525943
 8:19 pm on Aug 14, 2000 (gmt 0)

Does Yahoo's spider read robots.txt? I am beginning to wonder because we put a robots ban on another site a few weeks ago and the site was just dropped from Yahoo after getting a hit from xyz.yahoo.com last week. Is this connected or a coincidence?

 

JamesR




msg:1525944
 9:16 pm on Feb 28, 2001 (gmt 0)

*boing*

Anyone have any experience with Yahoo and robots.txt?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
WebmasterWorld ® and PubCon ® are a Registered Trademarks of Pubcon Inc.
© Pubcon Inc. 1996-2012 all rights reserved