Spider masking as a UCMore toolbar?

"GET /favicon.ico HTTP/1.1" 200 - "-" "UCmore"

[cyberkat]

Has anyone else seen these odd hits lately?
Earlier today, after receiving a four of these from different IP's, I decided to ban them.
The access log entries from the IP's only called a single "GET /favicon.ico HTTP/1.1" 200 - "-" "UCmore". Thus I banned the "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore". Since then the hits have remained steady, and more random IPs recorded.

The following is what was received in my access logs as of this posting:

1cust159.tnt41.lax7.da.uu.net - - [19/Oct/2003:01:50:44 -0400] "GET /favicon.ico HTTP/1.1" 200 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; MSN 8.0; MSN 8.5; MSNbBBYZ; MSNmen-us; MSNcIA)"

wcs2-cbus.nipr.mil - - [19/Oct/2003:09:10:30 -0400] "GET /favicon.ico HTTP/1.0" 200 3262 "-" "UCmore"

209-6-132-34.c3-0.wtr-ubr1.sbo-wtr.ma.cable.rcn.com - - [19/Oct/2003:10:06:59 -0400] "GET /favicon.ico HTTP/1.1" 200 - "-" "UCmore"

209-6-132-34.c3-0.wtr-ubr1.sbo-wtr.ma.cable.rcn.com - - [19/Oct/2003:10:27:01 -0400] "GET /favicon.ico HTTP/1.1" 200 - "-" "UCmore"

chello080109193054.1.graz.surfer.at - - [19/Oct/2003:10:36:05 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

cpe-24-161-129-59.hawaii.rr.com - - [19/Oct/2003:11:01:27 -0400] "GET /favicon.ico HTTP/1.1" 403 3262 "-" "UCmore"

host81-136-49-196.in-addr.btopenworld.com - - [19/Oct/2003:11:11:23 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

h59n3fls22o1073.bredband.comhem.se - - [19/Oct/2003:11:20:54 -0400] "GET /favicon.ico HTTP/1.1" 403 3262 "-" "UCmore"

lns-p19-7-82-65-237-86.adsl.proxad.net - - [19/Oct/2003:11:40:48 -0400] "GET /favicon.ico HTTP/1.1" 403 3262 "-" "UCmore"

213.51.84.39 - - [19/Oct/2003:12:15:23 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

81.203.219.85 - - [19/Oct/2003:12:24:08 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

81.203.219.85 - - [19/Oct/2003:12:26:54 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

24.24.41.54 - - [19/Oct/2003:12:59:34 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

81.99.46.96 - - [19/Oct/2003:13:02:56 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

200.82.89.253 - - [19/Oct/2003:13:18:50 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

209.8.204.84 - - [19/Oct/2003:14:27:47 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

209.8.204.84 - - [19/Oct/2003:14:28:36 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

220.226.43.128 - - [19/Oct/2003:14:33:10 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

200.84.174.10 - - [19/Oct/2003:15:13:06 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

200.29.158.107 - - [19/Oct/2003:15:16:05 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

[cyberkat]

17 more log entries to add to the my list above:

80.58.17.42 - - [19/Oct/2003:15:42:07 -0400] "GET /favicon.ico HTTP/1.0" 403 - "-" "UCmore"
213.54.62.233 - - [19/Oct/2003:16:04:15 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
162.84.113.180 - - [19/Oct/2003:16:29:17 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
80.58.9.172 - - [19/Oct/2003:16:45:13 -0400] "GET /favicon.ico HTTP/1.0" 403 - "-" "UCmore"
217.83.41.251 - - [19/Oct/2003:16:50:23 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
67.10.27.199 - - [19/Oct/2003:16:59:00 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
148.221.36.63 - - [19/Oct/2003:17:18:30 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
151.196.122.212 - - [19/Oct/2003:17:31:27 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
217.136.129.71 - - [19/Oct/2003:17:35:03 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
12.226.200.59 - - [19/Oct/2003:18:19:40 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
12.226.200.59 - - [19/Oct/2003:18:20:50 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
81.135.95.56 - - [19/Oct/2003:18:48:32 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
81.218.75.37 - - [19/Oct/2003:19:04:27 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
206.172.142.66 - - [19/Oct/2003:19:20:32 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
66.122.244.177 - - [19/Oct/2003:20:02:59 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
200.82.111.130 - - [19/Oct/2003:20:51:20 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
210.205.211.46 - - [19/Oct/2003:21:20:32 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

[claus]

One older thread, also mentioning the favicon:

[webmasterworld.com...]

You seem to get a lot of them though, and from odd places too. It seems that when you go to their homepage and do a search, then some of the results are shown with a little icon next to them, that could be the favicon.

[keyplyr]

Yes, tonight UCmore made 13 unique requests for favicon, all from different IPs.
No other rerquests were made from these IPs AFAIK.

195.92.168.168 - - [19/Oct/2003:03:43:34 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
217.209.220.19 - - [19/Oct/2003:04:56:44 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
200.193.25.32 - - [19/Oct/2003:06:38:59 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
213.250.59.77 - - [19/Oct/2003:06:41:50 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
80.140.44.176 - - [19/Oct/2003:07:36:25 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
213.60.84.19 - - [19/Oct/2003:07:54:02 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
64.166.125.38 - - [19/Oct/2003:10:00:43 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
142.59.36.218 - - [19/Oct/2003:11:03:30 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
24.205.25.120 - - [19/Oct/2003:11:07:49 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
24.45.208.211 - - [19/Oct/2003:11:39:14 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
80.58.41.46 - - [19/Oct/2003:11:56:21 -0700] "GET /favicon.ico HTTP/1.0" 200 318 "-" "UCmore"
12.254.232.224 - - [19/Oct/2003:13:26:15 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"
65.50.66.69 - - [19/Oct/2003:14:01:27 -0700] "GET /favicon.ico HTTP/1.1" 200 318 "-" "UCmore"

[claus]

I've employed a bit of research. I found this on the UCMore site:

Q. Why do I see icons next to some of the links?
A.Sites that have an icon appearing next to their link are using a well known mechanism called Favicon (www.favicon.com). In Internet Explorer 5.0 and above, Favicon allows you to customize the icon next to your bookmarked web site to be your own customized logo. UCmore's automatically detects this mechanism and displays the icon next to the corresponding link.
[ucmore.com...]

Now, i'm wondering - should i ban this behavior preemptively? That's something i only do in very rare cases. Requests for "favicon.ico" is a statistic that i monitor closely, as this tells me how many of my visitors that bookmark the site. So, if these requests are not real people making bookmarks then i really don't want to count them as such.

The question is: Is "the links" from the above quote equal to bookmarks or not. Well, this is what i found in the sourcecode of their search results on the site search page - next to the site listings:

<IMG (...) src="http://example.com/favicon.ico"></IMG>

However, if this was it, your requests would probably also show the following referrer string:

http*//ucmore.com/tool_search.asp?q=search-term

And it does not. So, this could still be bookmarks, but the toolbar itself seems to do no more than list these search results. These are "the links" from the above quote. This is where the favicons are fetched - that also explains the lack of referrer info and the different IP's. Well, i definitely don't want my favicon fetched if it's not a real bookmark, so i'll ban it.

It seems the best way to do this is to use a combination of the request and the user agent. I'll add an empty referrer string (or an "-") just to be sure. Here goes:

RewriteEngine on 
RewriteCond %{HTTP_USER_AGENT} ^UCmore$ [NC]
RewriteCond %{HTTP_REFERER} ^-?$
RewriteRule favicon\.ico$ - [F,L]

That'll do it. Only problem is that it will still make requests for that file, so i can't use the raw number of requests as an indicator of bookmarks anymore. I'll have to filter the logs and take out those requests being served a forbidden code in stead - as this is more work than needed, i definitely don't like that.

/claus

[cyberkat]

"Claus", thanks for the link of the past post. Interesting that it is labled a parasite.
And if "keyplyr" is also having the same entries. What is going on with the darn thing? I have, at the time of this post, a total of 57 hits of this kind in 28 hours. I have scanned through all my logs for the past 12 months and this UA has never appeared in my logs, until yesterday.

I have banned this due to the following:
1) Rogue IPs, all with UAs "UCmore"
2) All with one single(rarely a double) request of Get /favicon.ico
3) The quanity of times this has happened in a short period of time.

Since my two prior postings, I have been hit another 20 times. I add the following to the lists above:

216.195.29.42 - - [19/Oct/2003:21:58:49 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
203.109.73.60 - - [19/Oct/2003:22:27:25 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
63.101.13.182 - - [19/Oct/2003:22:45:22 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
24.202.139.246 - - [19/Oct/2003:22:55:26 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
68.113.116.87 - - [19/Oct/2003:23:31:47 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
68.113.92.78 - - [19/Oct/2003:23:33:41 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
67.31.169.98 - - [20/Oct/2003:00:50:22 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
65.32.89.117 - - [20/Oct/2003:02:22:33 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
200.76.168.97 - - [20/Oct/2003:05:18:59 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
200.76.168.97 - - [20/Oct/2003:06:09:57 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
62.42.76.2 - - [20/Oct/2003:06:11:29 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
217.210.235.2 - - [20/Oct/2003:06:24:19 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
210.10.172.242 - - [20/Oct/2003:07:02:14 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
217.80.197.87 - - [20/Oct/2003:07:38:34 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
209.4.43.140 - - [20/Oct/2003:08:35:38 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
195.121.210.207 - - [20/Oct/2003:08:39:13 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
68.169.48.101 - - [20/Oct/2003:09:09:24 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
209.4.43.140 - - [20/Oct/2003:09:15:20 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
80.202.173.209 - - [20/Oct/2003:09:28:36 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"
68.46.158.124 - - [20/Oct/2003:09:48:39 -0400] "GET /favicon.ico HTTP/1.1" 403 - "-" "UCmore"

[BlueSky]

You guys are showing only the log entries for UCmore. Some bots change their UA's as they crawl. Since this one is using IPs from a bunch of different nets, this may be difficult to even answer but are you seeing any unusual activity between these favicon.ico requests?

[claus]

>> 1) Rogue IPs, all with UAs "UCmore"

I wouldn't put it that way. The IP's are simply ordinary users that have the UCMore toolbar installed. I would not ban the IP's - only the combination "UCMore+favicon.ico+no referrer" (as above). It's not a bot or a spider.

It definitely is a parasite though, as it is requesting the favicon without a bookmark is being made, and even without a physical visit to the site by the toolbar user. If the user base of this tool increases it will lead to a lot of unnecessary bandwith drain.

What happens is that a user goes to some site. Then, the UCMore toolbar places a set of links to related sites in a special dropdown menu. In this this dropdown menu, they include your favicon.

The idea is okay, it's a bit alexa-like, but the effect is a potentially massive bandwith drain, as well as making the bookmark stats of this world even more unrealiable than they are, and filling the logfiles with useless entries. Imagine the effect if this "tool" should get a million users or so - it's not exactly a nice scenario.

I don't think these sideeffects are intended, but that does not make it any nicer. I haven't seen requests from it yet, but i suggest you send them an email and ask them to stop this behavior, along with the logfile entries. And ban it of course.

/claus

[keyplyr]

UCmore's hunger for favicons continues at my site today...

[cyberkat]

I am now totaling over 330 hits for the favicon.ico and increasing like wild fire. I only banned the UCMore, not the IPs. Other then user generated installs, I found this toolbar(UCMore)is also installed via drive-by-downloads on some domains other then the UCMORE.com website. So I guess the IPs are all innocent surfers infected by this.

After looking further into this intruder(UCMore), Pest Patrol had good information on this UCMore.
REF LINK: [pestpatrol.com...]
Pest Patrol States on this page under RISKS-Privacy Issues:, "Yes. Every URL opened is posted to the servers at users.ucmore.com together with a unique ID."
And AND.DOXDESK info: [doxdesk.com...]
After reading this,I would classify this UCMore as a bot using innocent people to do its dirty work.

Now wouldn't that be considered bandwidth thieft, in the terms of a person adding a gif to their website, being linked from your server? As they are doing with the favicon.ico file? That is a violation of our site's TOU.
But who do you complain to?

Currently looking into this parasite website tool (parasite.js & detector.js) It may help clean up at least some of the mess. I found it herehttp://www.associate-advocate.netfirms.com/scumware/solution.htm

The links on this page referring to the parasite.js download are errored, refer to domain and.doxdesk.com the correction is [doxdesk.com...]

[claus]

>> But who do you complain to?

You complain to the company, and if you are not satisfied with their reply, you complain to their ISP/host. Both are located in Israel it seems; a whois lookup will provide contact info. Afaik, this country is well known for certain types of software; UCMore is not exactly the "bad boy" of the class, so it just might be worthwhile to send them (both) a polite but firm "Cease and Desist" note.

The problem with detecting users of the Toolbar on your site is, that these users are not visiting your site. They are visiting some other site, and being on this other site, the toolbar downloads your favicon, as your site is in some way deemed "related" to the site these users are on.

/claus

[BlueSky]

I tried their search engine. A lot of sites listed do not have any img tags with favicon.ico. Since your logs don't show double entries (one for testing file existence and then one for user display) when they got 200's, perhaps they keep a database on which sites have favicon.ico.

I consider this bandwith theft since they're hotlinking. If after writing they won't stop their software from pulling this file on your site, why don't you (or ask your host to) block these requests at the firewall so they don't pollute your logs. Alternatively, you can tell these guys every time you get a hit from any user you'll bill them for bandwidth usage at the generous rate of say $100/hit.

[keyplyr]

I called it parasite-ware in this thread [webmasterworld.com] a while back. I feel it pushes the ethical standards quite a bit and was hoping it would get it's act together like Alexa (almost) did. Shame to ban unsuspecting users just because they have aquired a tool they feel is useful, and in turn loose traffic.

[BlueSky]

Preferably you don't want to ban the users just the software from pulling that particular file. Hopefully, the fix is easy for the company to do. If the visit rate from that toolbar is extremely low to none, which I suspect it is, for me it would not be worth having my logs polluted with all those extra favicon.ico calls. You can continue letting them thru if you want. That file is usually pretty small. For me, I wouldn't want the extra log entries because they would skew my stats and make it more work to find bad guys who are trying to mess around with my site. I could write a script to filter those out, but it irks me to do extra work just because a company was very thoughtless in designing their product.

[claus]

That's exactly it BlueSky

>> perhaps they keep a database on which sites have favicon.ico

Look at the sourcecode of those serps, they just do a javascript test to see if it's available or not, i don't remember the exact syntax, but it's a smart little trick.

/claus

[cyberkat]

Well the hits for the favicon with UA of that UCMore keep stacking high. You would think that now after over 500+ hits and the return of the 403 to the user.ucmore.server that thing would get a clue to delete it from its database. I hate stupid data collectors.
Yesterday I added parasite.js finder I found it here [doxdesk.com...]
Since my last post, I placed the script on the site and in the main page with a little notice "SPYWARE: Is it on your PC? Our main page now checks for hits from parasites including: UCmore, Comet Cursor, Gator, BonziBuddy, GoHip, Browser Hijacker, DownloadAccel, Flyswat, freeNsafe, to name a few of the 1,000+ lerking out there.
I got a thank you email today from a visitor stating that his pc is running alot faster when my site told him he was infected with DyFuCA, Xupiter, NewDotNet, Gator. He clicked on the Adaware link and cleaned the mess up.

Well, I hope this helps clean up some of the mess lerking around, people don't know they have on the system.

[claus]

My favicon request count for this month is now double that of last month. I find this v-e-r-y annoying, as this is not real visitors making real bookmarks.

Yesterday alone, i got these hits 99 times. I've seen a few HTTP/1.0 as well as 1.1

This is the bit of code that calls the favicon from their search page:

<TD class=desc_search width=46>&nbsp;<IMG style="MARGIN-BOTTOM: 12px" onerror="this.src='images/err.GIF'" src="http://example.com/favicon.ico" border=0></IMG></TD>

This code is only included in the HTML for sites that have a favicon, so UCMore must send out some kind of bot to determine if the favicon is available or not. I don't know if this bot has an "UCMore" User-Agent string, but i do hope so, as if it has, then we'll just have to ban that UA and wait for the bot to come around.

Neither the UCMore site nor the Effective-i site has information about robots.txt as far as i have seen. Their ISP seems to be http //www.barak.net.il/ so i assume that they have an email address like "abuse at barak.net.il" - otherwise they do have one starting with "hostmaster"

/claus

[BlueSky]

hostmaster? If Israel falls under RFCs, it should be postmaster.

Any system that includes an SMTP server supporting mail relaying or delivery MUST support the reserved mailbox "postmaster" as a case-insensitive local name. This postmaster address is not strictly necessary if the server always returns 554 on connection opening (as described in section 3.1). The requirement to accept mail for postmaster implies that RCPT commands which specify a mailbox for postmaster at any of the domains for which the SMTP server provides mail service, as well as the special case of "RCPT TO:<Postmaster>" (with no domain specification), MUST be supported.

[faqs.org...]

[claus]

i know, "h" is what their whois info says. Btw, i just checked and they seem to have a working abuse email address as well.

[laurent3227]

Got over 800 UCMore parasite lines yesterday.

The UCMore site is linked to
[effective-i.com...]
which offers UCMore as one of their "solutions".
This lot have names, phone numbers and email addies.

Has anyone found a way of purging the ucmore lines out of the logs?

Laurent

[LowLevel]

Here is the log line generated by the UCmore spider that searched for my favicon:

62.90.166.248 - - [30/Oct/2003:16:44:22 -0800] "GET /favicon.ico HTTP/1.0" 200 2238 "-" "Mozilla/4.0"

So, if you ban that IP (or the C class), probably UCmore will not put a link to your favicon file on the toolbars.