Suddenly Robots

Coincidence? Other bots arrive on heels of MSN launch

[timster]

Has anyone else noticed other bots showing increased interest in their sites since the MSN launch?

Before MSNBot found my little hobby site, GoogleBot and Yahoo just weren't interested in indexing my site. MSNBot was kind enough to index my site thoroughly, and now that it's live my site gets a little traffic: 50-100 visits a day.

In the past couple days Slurp and Googlebot have started indexing much more of my site (hundreds of pages instead of a handful). WISENutbot and AskJeeves are also indexing more.

Any ideas of how this could be more than coincidence?

[BillyS]

Not sure if this is a pattern, but Wisenut indexed around 10% of my pages (total of 600). In the past they grabbed around 2 - 3 each month.

Jeeves indexed the entire site for the first time. Previously they had only grabbed around 10%.

Perhaps it is because the site is only 8 months old, but I thought that everyone was getting more aggressive since the MSN beta.

[Kukenan]

That's the beauty of competition.

A more competetive search market can be a good thing for all.