homepage Welcome to WebmasterWorld Guest from 54.234.147.84
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
Forum Library, Charter, Moderators: mademetop

General Search Engine Marketing Issues Forum

    
Brand new website, popular topic
unlimited




msg:249208
 8:42 am on Aug 29, 2005 (gmt 0)

My brank new website, about jokes, funny pictures, games, etc. got online 2 days ago. I checked the log file, find google, yahoo, and msn bot came. google and yahoo bots crawled thousands pages, but why MSN bot only crawled robots.txt, index.html and rss.xml?

 

robotsdobetter




msg:249209
 6:33 pm on Aug 29, 2005 (gmt 0)

There's no telling why bots do the things they do sometimes. It's not like they have a mind like ours (For now anyway), that can learn as it goes. Sometimes they will spider 100s of web pages and other days only a few. Give it some time and it will get around to your other web pages. Just be happy it knows your site is there.

Gargen




msg:249210
 3:47 am on Aug 30, 2005 (gmt 0)

Once google just wouldnt leave site alone 50k pages in a single day that means it was spidering multiple times, it was madness msn has done a similar thing but not on the same level, you will see very soon SE like G and msn are very unpredictible

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved