Welcome to WebmasterWorld Guest from 50.16.52.237

Forum Moderators: open

Message Too Old, No Replies

Ask Teoma Stopped Crawling

Have you observed it?

     
5:58 pm on Jul 17, 2008 (gmt 0)

Junior Member

5+ Year Member

joined:Jan 7, 2008
posts:164
votes: 0


I realize this is probably specific to our site, however I wanted to see if anyone else has noticed a drop in crawling from ask since about the 16th of July? We are very well indexed in Ask but the bot has all but stopped visiting our site and without any sort of diagnostics from ask I am trying to figure out the cause.
6:48 pm on July 17, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


I have noticed that they have dramatically slowed down their crawling on my sites, but they have not stopped.

A second data point, for what it's worth...

Jim

7:09 pm on July 17, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


I still see them here and there.
8:18 pm on July 24, 2008 (gmt 0)

Junior Member

5+ Year Member

joined:Jan 7, 2008
posts:164
votes: 0


Does anyone know how I could better diagnose why this is occurring (email, ask tool, etc)? It's still going on after two weeks now and I have yet to pinpoint any reason (the only thing we changed around that time was a removal of GA for a test). No robot.txt issues, or weird status codes. Could this be some sort of penalty (has anyone seen one similar)? I would love to fix it, but I am unaware how they would figure out we fixed it if they stopped coming all together.

We still show very well in Ask but the number of pages indexed is steadily declining. Previously, we were getting roughly 200 teoma hits an hour.

I have submitted our site to their sitemap ping tool without success (of helping the situation).

9:11 pm on July 24, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0



Mozilla/5.0_(compatible;_Ask_Jeeves/Teoma;__http://about.ask.com/en/docs/about/webmasters.shtml)

Hmm, yes. Now MIA for several weeks on most sites. Not seen anywhere at all for the last week now.

9:23 pm on July 24, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Just today:

65.214.45.117 - - [24/Jul/2008:12:15:14 -0400] "GET /robots.txt HTTP/1.0" 200 2664 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)"

Grabbed four pages. RDNS is crawler4008.ask.com from an IP range leased from MCI.

Jim

9:32 pm on July 24, 2008 (gmt 0)

Junior Member

5+ Year Member

joined:Jan 7, 2008
posts:164
votes: 0


The last hit from Teoma we saw on our primary site was:
2008-07-15 09:10:06 65.214.45.148 Mozilla/5.0+(compatible;+Ask+Jeeves/Teoma;++http://about.ask.com/en/docs/about/webmasters.shtml)

Others sites saw it last many days before that.

3:22 am on July 25, 2008 (gmt 0)

Junior Member

5+ Year Member

joined:Jan 7, 2008
posts:164
votes: 0


An update, we had 9 hits from teoma over the last 3 hours. 2 for robots.txt and the rest for deep urls.
10:25 am on July 25, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Sept 29, 2000
posts: 12095
votes: 0


Ask 187+51 2.66MB 24 Jul 2008 - 14:54

The site only has about 50-60 pages altogether, and they seem to hit it frequently. (That's for the whole month, last visit on the 24th).

[edited by: Marcia at 10:28 am (utc) on July 25, 2008]

11:06 pm on July 28, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


They came back on July 24th, just a few minutes after I posted my comments above about being missing for a week or two.
2:16 pm on Aug 1, 2008 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 21, 2001
posts:2489
votes: 0


maybe the ask/teoma crewe have jumped google, and now can do hand edits of websites on demand to users of web dev type forums, becuase they have worked out who is each website.

They wish - i'm seeing very sparodic crawling, certainly slowing down.

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members