homepage Welcome to WebmasterWorld Guest from 54.205.241.107
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
qestion
to refuse approch
mimmo5

10+ Year Member



 
Msg#: 697 posted 9:28 am on May 23, 2001 (gmt 0)

If the approch in a directory like [moda.online.it...] has refused

how the spider can index the page
Make patience for my Englisc
ciao

 

Air

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 697 posted 1:00 pm on May 23, 2001 (gmt 0)

It cannot, spiders cannot index protected directories.

mimmo5

10+ Year Member



 
Msg#: 697 posted 7:55 am on May 24, 2001 (gmt 0)

ok then,why this page [moda.online.it...] is in 2 place in altavista

Air

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 697 posted 12:29 pm on May 24, 2001 (gmt 0)

Yes that is what will happen, if the spider is presented the "403" page, which it indexes, but that is not the content that is being protected. Maybe I mis-understand your question.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved