homepage Welcome to WebmasterWorld Guest from 54.227.215.140
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
Forum Library, Charter, Moderators: martinibuster

Yahoo Search Engine and Directory Forum

    
New Domain - Slurp only visits index and robots.txt
onlineleben

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3662 posted 7:15 pm on Nov 8, 2005 (gmt 0)

Asmentioned in the title, my new domain gets visited by Slurp, but only the index.html and the robots.txt get hit.
I have links to inner pages on the index.html which are followed by other spiders (msnbot, googlebot) and the robots.txt isn't disallowing slurp.
What could be the reason for this "ignorance"?
It seems not to be technical.
any help is veryy much appreciated.
Thanks

 

hmatisse

10+ Year Member



 
Msg#: 3662 posted 4:12 pm on Nov 9, 2005 (gmt 0)

Maybe they think that if Slurp ignores your pages for long enough, you'll sign up for SiteMatch. :)

aris1970

10+ Year Member



 
Msg#: 3662 posted 5:20 pm on Nov 12, 2005 (gmt 0)

I had exactly the same problem for a new site that I launched last February. It was fully crawled by Yahoo only after 6 months. Before that, Yahoo crawled only the home page.

It's like the sandbox effect in Google for new sites.

Tigrou

10+ Year Member



 
Msg#: 3662 posted 6:16 pm on Nov 12, 2005 (gmt 0)

None of my sites (old or new) were DeepCrawled in Oct. Check your stats today, they seem to have started crawling again.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved