|t-online.de german isp that spiders?|
| 7:07 pm on Apr 9, 2001 (gmt 0)|
This one got me yesterday.
Not too many visits, and the log entries are weird, as though it was making 2 requests in the same second. I believe t-online.de is an access provider, and it looks like they use the german infoseek for their search. Anybody else got more info on this?
| 10:27 pm on Apr 9, 2001 (gmt 0)|
May I respectfully draw your attention to the lively discussion on German search engines in the European forum, which has been split 4 times already and where T-Online and its search engine Infoseek.de (of which they own 25%) is one of the hottest topics. The news that they are spidering will be welcome in my part of the world, believe me.
Jan - your friendly European moderator.
| 2:00 pm on Apr 10, 2001 (gmt 0)|
Ahh, you spotted my American ignorance! How shameful of me to display it in the open, for everybody to see!
I wish to humbly apologize for ignoring my European brethren, I will be more consciencious in the future, and endevour to visit this forum.
| 4:00 pm on Apr 10, 2001 (gmt 0)|
Do you have listings in infoseek.de? They have in the past respidered already indexed pages. Activity has been very low though for the last few months. It has been one of the major SEs for Germany, but for the history and the problems of infoseek.de you might want to go to the european fo.....okay.
I could not find that "spider" (if it really is one) in my logs. You might want to ask some of the more experienced members in the eur....yes.
| 9:44 am on Apr 23, 2001 (gmt 0)|
Marvin/1.0 is the new spider from Infoseek.de because the old one (infoseek-sidewinder) is not fast enough.
The new one hits your site up to ten times a second, and it does a deep crawl (not that deep like FAST or Google, but it does).
Unfortunately IS has many problems with its index, it DOES need new hardware, the place within its db is absolutely full...I directly heard this from a employee of IS.
So Marvin spiders the net, but itīs not possible to get this new pages into the index of IS. Strange.
That could be a reason why T-Online donīt want to continue the partnership with IS: as an owner of IS T-Online has to pay a lot for the new hardware, and itīs well less expensive to pay Google for its results.