homepage Welcome to WebmasterWorld Guest from 54.159.239.248
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Ask - Teoma
Forum Library, Charter, Moderator: open

Ask - Teoma Forum

    
Everything crawled twice (at almost the same time)
Is Ask Jeeves' bot stupid?
luma

10+ Year Member



 
Msg#: 141 posted 8:26 pm on Jun 28, 2002 (gmt 0)

On 26/Jun/2002 some of my pages were crawled by Ask Jeeves. But why do they get everything twice within two minutes? I'd rather they'd index all of my site once instead of half of my site twice. Okay, same load but half the value. ;)

 

caine

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 141 posted 2:03 pm on Jul 1, 2002 (gmt 0)

not sure why they would:

1. crawl a selection of pages
2. crawl them twice

was it exactly the same bot ?

luma

10+ Year Member



 
Msg#: 141 posted 10:35 am on Jul 11, 2002 (gmt 0)

It was exaclty the same bot ("Mozilla/2.0 (compatible; Ask Jeeves)"). It took 54 pages (27 pages times 2).

They got exaclty the same 27 unique pages on July, 5th, so this seems fixed. Still, I wouldn't mind them getting all of my site.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Ask - Teoma
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved