homepage Welcome to WebmasterWorld Guest from 54.161.185.244
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
robots.txt question
I asked whether does this, but the engine spider ip address has any to use
ericchui

10+ Year Member



 
Msg#: 495 posted 9:53 am on Nov 21, 2004 (gmt 0)

# FULL access (yahoo)
User-agent: Slurp
Disallow: cgi

 

webboy1

10+ Year Member



 
Msg#: 495 posted 11:18 am on Nov 22, 2004 (gmt 0)

Not to sure what you are asking here! Sorry! Can you explain it a little clearer please.

Regards,
Webboy

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved