homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
Forum Library, Charter, Moderators: mademetop

General Search Engine Marketing Issues Forum

Brand new website, popular topic

 8:42 am on Aug 29, 2005 (gmt 0)

My brank new website, about jokes, funny pictures, games, etc. got online 2 days ago. I checked the log file, find google, yahoo, and msn bot came. google and yahoo bots crawled thousands pages, but why MSN bot only crawled robots.txt, index.html and rss.xml?



 6:33 pm on Aug 29, 2005 (gmt 0)

There's no telling why bots do the things they do sometimes. It's not like they have a mind like ours (For now anyway), that can learn as it goes. Sometimes they will spider 100s of web pages and other days only a few. Give it some time and it will get around to your other web pages. Just be happy it knows your site is there.


 3:47 am on Aug 30, 2005 (gmt 0)

Once google just wouldnt leave site alone 50k pages in a single day that means it was spidering multiple times, it was madness msn has done a similar thing but not on the same level, you will see very soon SE like G and msn are very unpredictible

Global Options:
 top home search open messages active posts  

Home / Forums Index / Marketing and Biz Dev / General Search Engine Marketing Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved