Question about spiders

how do you know who?

[Beach_lady]

How do you know who is a spider that should be there and the ones that should be avoided? I have "FunWebProducts" on my site everyday. I also have these:
64.12.96.75 cgi-bin/hello.cgi" "Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; .NET CLR 1.1.4322)"

64.12.96.236 cgi-auth/webshell" "Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; .NET CLR 1.1.4322)"
130.126.113.8 HTTP/1.1" 200 8814 "-" "Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.7b"

210.50.53.118 HTTP/1.1" 200 4084 "http://mamma26.mamma.com/Mamma?cookie=1075882204-0JKUGHOFHKNVLJIXAARSE&query=beach+rules&lang=1&qtype=0&rpp=15&cb=Mamma&index=31" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
207.86.191.101 cgi-auth/webshell" "Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; .NET CLR 1.1.4322)"

(compatible; MSIE 5.5; Windows NT 4.0; T312461)"
(compatible; MSIE 6.0; Windows NT 5.1; Nothing But Net, Inc. v8.02; .NET CLR 1.1.4322; MSN 6.1; MSNbMSFT; MSNmen-us; MSNc00; v5m)"
(compatible; MSIE 6.0; Windows 98; OptusNetCable-02; FunWebProducts)"

"Scooter/3.3.vscooter"
{1F1CFB84-9559-4539-BC6B-E6AD6ACF2373}; .NET CLR 1.1.4322)"

NET CLR 1.1.4322

"Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; .NET CLR 1.0.3705)"

(Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113"

"Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt)"

(compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; .NET CLR 1.0.3705; MSN 8.0; MSN 8.5; MSNbVZ02; MSNmen-us; MSNcOTH)"

compatible; MSIE 5.5; Windows 98; {starpower})"

(compatible; MSIE 6.0; MSNIA; Windows 98; sureseeker.com; H010818)"

(compatible; MSIE 6.0; Windows 98; Win 9x 4.90; Hotbar 3.0)"
(Slurp/cat; [e-mail]; [inktomi.com...]
(compatible; MSIE 6.0; Windows NT 5.1; Avant Browser [avantbrowser.com]; FunWebProducts; .NET CLR 1.1.4322; MSIECrawler)"

(compatible; MSIE 6.0; Windows 98; AT&T CSM6.0)"
HTTP/1.0" 200 28055 "-" "Scooter/3.3_SF"
HTTP/1.1" 200 24964 "http://userfiles.snuttis.se/80000/82844/?" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; FunWebProducts)"
(compatible; MSIE 6.0; CS 2000 6.0; Wal-Mart Connect 6.0; Windows NT 5.1; FunWebProducts)"
"Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; Hot Lingo 2.0)"
compatible; MSIE 6.0; Windows NT 5.1; EPIX-IEAK)"
(compatible; MSIE 5.01; Windows NT 5.0; {9E5A6E11-319A-4CF8-9D06-879A6A80471C})"
(compatible; MSIE 6.0; Windows NT 5.1; ESB{E57472A6-9547-4E2D-AC19-B8C8AD4B1DB6})"

(compatible; MSIE 6.0; Windows NT 5.1; AT&T CSM6.0)"

Are any of these spiders? And I have not seen anything of a Googlebot. I have found references to where there was a google search with the search terms but nothing that I feel is a spider. How often do they spider sites?

I'd appreciate any help. Also I know there is a thread somewhere that tells what spiders to block and how to block them with the robot code.
Thanks everyone.

[adfree]

Look for updated spider lists with Google or any other Se. You'll find nice sites that keep you up-to-date.
Jens

[keyplyr]

Beach_lady, it just takes a lot of reading, watching your logs and of course 'asking questions.' WebmasterWorld is a wealth of information. Search these archived forums for almost anything related to webmastering. Whenever I see traffic for an unknown User Agent or spider, I usually search Google and include webmasterworld as a search term to see if it was discussed here first. 9 outa 10 it has been.

[Beach_lady]

Does anyone know what Windows XP Hotfix is?

[jdMorgan]

Google knows [google.com].

The advice in the discussion shown as the first search result is good. Go to support.microsoft.com, and type in the "Q" number you see displayed along with each "HotFix" and it will tell you what that particular security patch does.

Jim