homepage Welcome to WebmasterWorld Guest from 54.226.161.112
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
qestion
to refuse approch
mimmo5




msg:398282
 9:28 am on May 23, 2001 (gmt 0)

If the approch in a directory like [moda.online.it...] has refused

how the spider can index the page
Make patience for my Englisc
ciao

 

Air




msg:398283
 1:00 pm on May 23, 2001 (gmt 0)

It cannot, spiders cannot index protected directories.

mimmo5




msg:398284
 7:55 am on May 24, 2001 (gmt 0)

ok then,why this page [moda.online.it...] is in 2 place in altavista

Air




msg:398285
 12:29 pm on May 24, 2001 (gmt 0)

Yes that is what will happen, if the spider is presented the "403" page, which it indexes, but that is not the content that is being protected. Maybe I mis-understand your question.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved