homepage Welcome to WebmasterWorld Guest from 54.167.179.48
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Ask - Teoma
Forum Library, Charter, Moderator: open

Ask - Teoma Forum

    
AskJeeves
Crawling webpages like crazy.
PhraSEOlogy

10+ Year Member



 
Msg#: 426 posted 3:40 am on Jul 3, 2004 (gmt 0)

Has anyone noticed that AskJeeves/Teoma bot has been crawling like crazy. My sites have Jeeves as the top bot crawling pages (ususally google - who is crawling like a snail).

Is this just an anomalie or have they ramped up their bot?

 

PatrickDeese

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 426 posted 3:47 am on Jul 3, 2004 (gmt 0)

I would speculate that the their stock tumble from MS's preview search engine release has turned up the heat a bit.

AJ is welcome to crawl my sites as much as they want - but I would prefer human visits from AJ to their bot.

steveb

WebmasterWorld Senior Member steveb us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 426 posted 3:58 am on Jul 3, 2004 (gmt 0)

Until I see the bot this year, or until I see the other 98% of my competitors sites in the Jeeves index, I'm tempted to consider these crawling like crazy posts to be the equivalent of Elvis sightings.

PhraSEOlogy

10+ Year Member



 
Msg#: 426 posted 4:12 am on Jul 3, 2004 (gmt 0)

Yep - I would agree with the Elvis analogy. But I got visited across multiple sites and a new site got 2276 pages requested in a couple of hours. That is probably more pages that have ever been requested by AJ bot in the past two years on all my sites.

Probably wont bring any traffic though!

asinah

10+ Year Member



 
Msg#: 426 posted 12:03 pm on Jul 3, 2004 (gmt 0)

AJ is crawling around 400,000 pages a month and we do get some good traffic from AJ.

steveb

WebmasterWorld Senior Member steveb us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 426 posted 8:56 am on Jul 11, 2004 (gmt 0)

<faint>

The number of pages in the index for one of my sites just went from one, where it has been for many months, to a few hundred. Less than half, but still about 250 times better.

I don't know if this represents anything for the Internet as a whole, but it is good to see.

Dayo_UK

10+ Year Member



 
Msg#: 426 posted 9:21 am on Jul 11, 2004 (gmt 0)

Yay - Jeeves update.

I think the crawl was about a week ago too - very good turnaround.

Jeeves is easiest my second biggest referrer behind Google.

I just wish Yahoo would get its act together.

Swanson

10+ Year Member



 
Msg#: 426 posted 11:34 pm on Jul 12, 2004 (gmt 0)

There was definately an update, traffic to some of my domains has increased by 10 times usual AJ levels.

New crawling has continued - normally updates occur on the 18th (ish) of every month according to my traffic logs.

b0rdslide

10+ Year Member



 
Msg#: 426 posted 11:50 am on Jul 16, 2004 (gmt 0)

How is it possible to find out how many pages for a particular site are in the index?

Thanks,
Kev

Dayo_UK

10+ Year Member



 
Msg#: 426 posted 9:27 pm on Jul 19, 2004 (gmt 0)

Hi Kev

inurl:www.yourdomain.com

should do the trick.

Dayo

yowza

10+ Year Member



 
Msg#: 426 posted 10:26 pm on Jul 19, 2004 (gmt 0)

Last month AJ visited my new PR3 site and spidered 700 pages in one visit.

I haven't seen AJ since....

steveb

WebmasterWorld Senior Member steveb us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 426 posted 9:33 am on Jul 20, 2004 (gmt 0)

<back to reality>

The few hundred pages jeeves managed to finally find from one of my domains maneged to last all of nine whole days. Now POOF they are all gone again.

I guess Jeeves has all that cash on hand because they don't spend 50 cents to keep their hamster-powered search engine from constantly collapsing.

Checking several other sites with just one pfi page, it looks like they have completely removed all pages except the pfi ones.

andrew_m

10+ Year Member



 
Msg#: 426 posted 10:08 am on Jul 20, 2004 (gmt 0)

They definitely crawl like crazy. That is they consume about 40% of my traffic for last month and half at least. Average about 65000 hits a day.

And, wow, I get something like 0.04% of human traffic from them.

Gets on my nerves lately to tell you the truth. If there is someone from AJ/teoma here I'd love to hear them..

RichD

10+ Year Member



 
Msg#: 426 posted 10:43 am on Jul 20, 2004 (gmt 0)

site:www.domain.com domain seems to provide better results than inurl:www.domain.com - see the results for inurl:www.amazon.co.uk on ask.co.uk for example.

Ask Jeeves has been crawling a few of my sites heavily for the last few months, so much so I had to upgrade my hosting (1.3million hits so far this month on one site). But Ask Jeeves does now provide 33% of the volume of visits that Google does, so it has been worth it for me.

steveb

WebmasterWorld Senior Member steveb us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 426 posted 11:15 am on Jul 20, 2004 (gmt 0)

And now poof, pages are back in.

(I searched for text on a bunch of the these missing pages, not just the site and inurl commands.)

CodeJockey

10+ Year Member



 
Msg#: 426 posted 4:32 pm on Jul 20, 2004 (gmt 0)

Teoma has been lighting us up for about 30,000 combined hits a day for a while now. While the traffic isn't as nice as Yahoo or MSN, it's about equal to Google. The bot is reasonably polite. Hard to turn it off when it's actually returning something.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Ask - Teoma
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved