redzone
msg:32397 | 2:44 am on Dec 6, 2000 (gmt 0) |
DH isn't using the "Direct Hit" Grabber User Agent any longer. Since they have been acquired by AskJeeves, we have been seeing a UA of: Mozilla/2.0 (compatible; Ask Jeeves) IP Class C's of:
209.67.252
216.34.121 coming in to crawl.... Hope this helps...
|
Cees
msg:32398 | 2:51 am on Dec 6, 2000 (gmt 0) |
redzone,
thanks I'll be looking for those ip's.
How long does it usually take after submission before site is crawled and than included?
|
redzone
msg:32399 | 2:48 pm on Dec 7, 2000 (gmt 0) |
Could be as long as 90-120 days, if you get in.. DH only shows top 200 results, so I've never been able to figure out, if your not in the top 200, are you possibly still in the index, and just buried? Similar to AV?
|
msgraph
msg:32400 | 3:51 pm on Dec 7, 2000 (gmt 0) |
Rumor has it that you have to be listed in Inktomi before you submit to Direct Hit in order to receive their spider. I'll try to find that info and post a link. Maybe things have changed recently. I've pretty much found it to be true. Of all the sites I received the Direct Hit Grabber on, all of them were listed in Inktomi results. Redzone: I received DH Grabber yesterday on a site that I have submitted to PT. (Not sure if there is any relation) I just wonder because DH only went after the pages I submitted to PT and nothing else. I didn't see the robot in any other domains either UA: Mozilla/4.06 [en] (Direct Hit Grabber) IP:
209.67.252.214
209.67.252.211 Both came off of Exodus servers so I presume they are from DH
|
redzone
msg:32401 | 3:42 pm on Dec 8, 2000 (gmt 0) |
msgraph,
we are still seeing the "ask jeeves" UA coming from that same class C as you mention, without Inktomi results in place for the URL.. Will be interesting to analyze this over the next 60 days. This was the first I had heard of the Inktomi listing related to "DH Grabber" UA's.. WE just assumed that DH had changed their crawler based on the buy out..... I'll keep you posted if our "ask jeeves" crawls from DH, get listed in their index over the next 30 days...
|
msgraph
msg:32402 | 3:59 pm on Dec 8, 2000 (gmt 0) |
I'll keep an eye out too. But yeah it gets to be fun sometimes wondering about all of this. They all interweave everything so often that I begin to think that ALL these search engines are owned by one big company who likes to play games with our heads.
|
|
