homepage Welcome to WebmasterWorld Guest from 54.198.33.96
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Ask - Teoma
Forum Library, Charter, Moderator: open

Ask - Teoma Forum

    
AskJeeves
Crawling webpages like crazy.
PhraSEOlogy




msg:32510
 3:40 am on Jul 3, 2004 (gmt 0)

Has anyone noticed that AskJeeves/Teoma bot has been crawling like crazy. My sites have Jeeves as the top bot crawling pages (ususally google - who is crawling like a snail).

Is this just an anomalie or have they ramped up their bot?

 

PatrickDeese




msg:32511
 3:47 am on Jul 3, 2004 (gmt 0)

I would speculate that the their stock tumble from MS's preview search engine release has turned up the heat a bit.

AJ is welcome to crawl my sites as much as they want - but I would prefer human visits from AJ to their bot.

steveb




msg:32512
 3:58 am on Jul 3, 2004 (gmt 0)

Until I see the bot this year, or until I see the other 98% of my competitors sites in the Jeeves index, I'm tempted to consider these crawling like crazy posts to be the equivalent of Elvis sightings.

PhraSEOlogy




msg:32513
 4:12 am on Jul 3, 2004 (gmt 0)

Yep - I would agree with the Elvis analogy. But I got visited across multiple sites and a new site got 2276 pages requested in a couple of hours. That is probably more pages that have ever been requested by AJ bot in the past two years on all my sites.

Probably wont bring any traffic though!

asinah




msg:32514
 12:03 pm on Jul 3, 2004 (gmt 0)

AJ is crawling around 400,000 pages a month and we do get some good traffic from AJ.

steveb




msg:32515
 8:56 am on Jul 11, 2004 (gmt 0)

<faint>

The number of pages in the index for one of my sites just went from one, where it has been for many months, to a few hundred. Less than half, but still about 250 times better.

I don't know if this represents anything for the Internet as a whole, but it is good to see.

Dayo_UK




msg:32516
 9:21 am on Jul 11, 2004 (gmt 0)

Yay - Jeeves update.

I think the crawl was about a week ago too - very good turnaround.

Jeeves is easiest my second biggest referrer behind Google.

I just wish Yahoo would get its act together.

Swanson




msg:32517
 11:34 pm on Jul 12, 2004 (gmt 0)

There was definately an update, traffic to some of my domains has increased by 10 times usual AJ levels.

New crawling has continued - normally updates occur on the 18th (ish) of every month according to my traffic logs.

b0rdslide




msg:32518
 11:50 am on Jul 16, 2004 (gmt 0)

How is it possible to find out how many pages for a particular site are in the index?

Thanks,
Kev

Dayo_UK




msg:32519
 9:27 pm on Jul 19, 2004 (gmt 0)

Hi Kev

inurl:www.yourdomain.com

should do the trick.

Dayo

yowza




msg:32520
 10:26 pm on Jul 19, 2004 (gmt 0)

Last month AJ visited my new PR3 site and spidered 700 pages in one visit.

I haven't seen AJ since....

steveb




msg:32521
 9:33 am on Jul 20, 2004 (gmt 0)

<back to reality>

The few hundred pages jeeves managed to finally find from one of my domains maneged to last all of nine whole days. Now POOF they are all gone again.

I guess Jeeves has all that cash on hand because they don't spend 50 cents to keep their hamster-powered search engine from constantly collapsing.

Checking several other sites with just one pfi page, it looks like they have completely removed all pages except the pfi ones.

andrew_m




msg:32522
 10:08 am on Jul 20, 2004 (gmt 0)

They definitely crawl like crazy. That is they consume about 40% of my traffic for last month and half at least. Average about 65000 hits a day.

And, wow, I get something like 0.04% of human traffic from them.

Gets on my nerves lately to tell you the truth. If there is someone from AJ/teoma here I'd love to hear them..

RichD




msg:32523
 10:43 am on Jul 20, 2004 (gmt 0)

site:www.domain.com domain seems to provide better results than inurl:www.domain.com - see the results for inurl:www.amazon.co.uk on ask.co.uk for example.

Ask Jeeves has been crawling a few of my sites heavily for the last few months, so much so I had to upgrade my hosting (1.3million hits so far this month on one site). But Ask Jeeves does now provide 33% of the volume of visits that Google does, so it has been worth it for me.

steveb




msg:32524
 11:15 am on Jul 20, 2004 (gmt 0)

And now poof, pages are back in.

(I searched for text on a bunch of the these missing pages, not just the site and inurl commands.)

CodeJockey




msg:32525
 4:32 pm on Jul 20, 2004 (gmt 0)

Teoma has been lighting us up for about 30,000 combined hits a day for a while now. While the traffic isn't as nice as Yahoo or MSN, it's about equal to Google. The bot is reasonably polite. Hard to turn it off when it's actually returning something.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Ask - Teoma
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved