Just put up a new site and I thought I'd have a look at which bots came in first.
First in: Ask/Teoma - 2 hours after I put the site up and had a good crawl around the full site. Second: MSN - about 4 hours Third: Google - about 12 hours, looked for robots.txt then left
(Also a few spambots have been around)
No sign of Yahoo yet.
Anyway, I thought this was interesting to see that Ask came by so fast, since Ask is listing and ranking my sites very fast and very well at the moment. I don't have PFI on any sites, but Ask is definitely my best friend at the moment.
Anyone else getting visited by Ask/Teoma before anything else?
It's a new site, not just a new page. And is approx 20 hours old at the time of posting.
The new site has only one inbound link at the moment (so far as I know) and that's from a directory page on my main site which is constantly and heavily spidered.
I was interested to see which spiders came in first and how much of the site they crawled. Ask has been doing really well all round for me at the moment, but I was still surprised to see a full crawl from the bot that quickly.
I'm going to keep an eye on this one and see when it gets indexed and ranked in various SEs.
Wow. One of my sites has been around since like 4 or 5 months, it ranks well on Google and Yahoo, receives a decent amount of traffic, but Ask/Teoma is the only one search engine to not have fetched it yet!
And as there's no URL submissions page, I can't ping them to.
There is a way of course, PFI, but I'd rather spend money on advertising :)