homepage Welcome to WebmasterWorld Guest from 23.22.173.58
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Infoseek Sidewinder
Some new spider flavors
Fusioneer

10+ Year Member



 
Msg#: 293 posted 3:26 pm on Jan 17, 2001 (gmt 0)

The following are all new spiders/UA's in January alone (for my sites anyway). Just keeping everybody up to date if they have not seen these (?)

Note Infoseek Sidewinder - never seen this before.

2001-01-17 08:28:17 cca260c3.infoseek.com - GET /robots.txt 200 0 350 196 10 HTTP/1.0 InfoSeek+Sidewinder/0.9 - -

2001-01-16 21:55:43 aesop.nb.no - GET /robots.txt 200 0 350 205 30 HTTP/1.0 FAST-WebCrawler/2.2-pre-cvs+(fred.moerman@kulturnett.no;+http://www.kulturnett.no) - -

2001-01-15 12:32:48 www.domanova.co.uk - GET /robots.txt 200 0 350 147 0 HTTP/1.0 Jack - -

2001-01-15 07:05:12 207.14.232.20 - GET /robots.txt 200 0 350 111 10 HTTP/1.0 PCBeaconBot/1.0 - -

 

han solo

10+ Year Member



 
Msg#: 293 posted 5:11 pm on Jan 18, 2001 (gmt 0)

Let's see:

PCbeaconbot-not infoseek. I wouldn't serve them your content, look at the site associated with this bot, and you can check things out from there. Search for the UA, I won't give them the courtesy of dropping their home url. Just use your favorite engine, and I'm sure you'll find what this one is about. Or I can email you in your stickymail, if you like.

Jack- This one still is a mystery to me, but I've yet to see any proof of it relating to infoseek. It's hit me from some isp IP's before, and I don't let it have anything...but have you found a connection to Infoseek? I'd be interested in checking that out.

The other UA seems right, though. Thanks for the heads up.

Cheers,
Han Solo

Fusioneer

10+ Year Member



 
Msg#: 293 posted 12:37 am on Jan 19, 2001 (gmt 0)

Sidewinder UA is defintely from Infoseek, but the others are from different sites.

Domanova.co.uk is a very simple looking UK search engine producing pretty mediocre results. At this stage I'm not including it in my cloaking (well the spider is my db but disabled).

Likewise for pcbeacon.com which is a "business-to-business' directory.

The weird one is aesop.nb.no with "FAST" included in the UA which looks like it came of the Norwegian National Library (Nasjonal Biblioteket). I'm not serving this one cloaked content either.

- Fusioneer

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved