Garbage user agent srings and changing IP addreses

Impossible to block?

[bouncybunny]

I have been getting repeated visits from 'something', but the only trace they leave are strings such as "jkgfyutfcjjyrxddertsxtufy7crtexr". The string of charactors keeps changing, as do the IP addesses (which vary from eastern Europse to the US).

Can anything be done?

[Ocean10000]

I been seeing these type of bots comeing to my pages for the last year and a half or so. I know Gary also been seeing these buggers for alittle longer then I have.

The way I combat these random named UA them is:

(1) Test Ip, in bad range, give 403 error.

(2) I test the UA against Known browser UA Profiles, if it matches a known Browser profile I do nothing.

(3) Do a lookup of common words often found in UA's, if it matches a known word I do nothing.

(4) If it gets this far I just give it a 403 error.

The hardest part is just keeping an updated Known browser UA Profiles. I maintain an up todate browser agent list for Asp.Net applications & Gary also maintains ones for Asp & Php based web app's.

[Mokita]

bouncybunny wrote:

Can anything be done?

If you are hosted on an Apache server and can edit or create an .htaccess file, I recommend trying Jim's method, for random-letter User-agents:

[webmasterworld.com...]

I use his code, and it works well for me.

[volatilegx]

That's a great post by Jim, well worth bookmarking. Genius!

[Ocean10000]

Well I don't know how much test data everyone has to test there versions of the perfect .htaccess files. Here are some Random User-agent that I have seen in the last year, I doubt all of them would be caught buy the versions I have seen so far.

--Start of List --
0jshpwtn ssswhcm0 bjmjvybulo
0qdvpnatghpq0perueygpphnnvui0jmfb
0xuyebsopebus e nkfhtpv0ujnic
1bwvvyijiWmk axrk11nnWny1 gniWW
1hpvjjbvtxfyyfhdixwicnm1ekvxh1xiix
1rteffaiqsdxbxo x mqejjytbwrxafneaeybb
2adtrmwWohoWuWckwnvtfk2xux
4ecqbc feoeusxicbwpsxc difs4qcao
4lwmlyxtpwlpAkfplolAkqjcx4
5Ji5fpppeudtwdxmjbjtedxthruwJyfb
6toacpfrlxofmdyxmXfipa
7cey twyhwveGymbxte7ffb
8xwfylst8wnfsfrruockCob88hycqvos
9cvpplkcSkoyqqlqrksjekp9
acxnav qivtgro9tiffrk9a
ahwrXehlqjaXibarggwdaxcvbsmsXn
aKoiwmcowKencicvacoiyiKmuuiuwv
akrwrdlai uathjnnffloJwweidkft
amymflM9po axxadnjesagsm qkupocwooMspn
arkxnvaayoxetkcdxcrm xnfceken2
aseicdgjegsrlY5dxrcfl
avqhk ai hwt7D7aq o
b CtmtlCdvfmwlesddckwsyx kyfxvn
BafusilsiB sugwkgrswi 2gdcolq
bbnkrltupppkol dHnvmpgsoi auoymH
bcjtptivldvwvRgpflRjwRkicdg
bf ohtBnsp7psga ByfB7i7o
bieldnogvlqrogtjbql2kgqkigtwibjolmr
bIrsksnamIehnsdybuiixenlavsbux
bkmvfofecrrn2tfd jijtp2peksKavvsvKqf
bnbbadugdqwhiL qmm5fLugm Li5sy5mmxk
bpbwoeadslesvkpdemoytj oe cgsn
bpriDhkuqnjhi6dlbuimdh6aDil e
bSawyxavb ihccqebpxceqwgSxai
btnhEviatf8iwsantwk8h
c exracbdtwm3mm vrpnevd3qp
chaxcgp4uwqgwafxaocxiu
Chxedwwiaq0maepw0va0csawiCfeidxhdlbylt
cpgjJbwt udqlikndqnkf
ctjlqvgvnfhlpjwsycoodxhqhhxw db
cuboomtDsepjlusap 9ivD
cVsVodVoosuVedi9gdspf ji9wypnrnmodwm
cvVpyxVbpl esbyoggddn
cxehrlwCtxpxctwdqyciypq
cxts cmolnkfkxksqkbwyfgsPbthtgmmyhql u
Cyit1twofofCtux wquyrkdfCffaema
deaeYeaqxrexaaYvqejb xoiYYr
developer
dfiwnd 5kggppvyvsx dc5kmpkhxhpgpsbvfy
dM1ksowtjqs umqdsbbryyuk bc dqscyvid
DmrjuscDtmDpyduhwoDown
dpbPfxwu4ikgvppufqfeu4s
dpjkkkeqbq6ht khrqtEkaqkfs
dqcpqdnwirh psuhcgogpr
dquni8rclpkrRi8RRilRahdbnekscesdf 8k
dsfadx 1xt u11kwwmuqddqsetaff ehwsxk
dvnfecwwfcvsxlpceadhsdnuw
DvyvxexkcjqDsqqmavDsdgs
E0ocrjxwwEtyyjdEvluvbksx
ed0i0yhqhyfddsfpsematmrmmefmynenhiltqp
ediAvjtsokgsrxjpjijgxg n wfvftvex lan
eduqntxoi nrgvugsuof
egqGdxeilonrGqtoywmotiyc1ytwfofbin
ehboidqtiubscqc8aaahm
eiJqJaJpqqmro ileosqmccb7dgaymu e
entftakehnyhxutn3ws 3OrvygkgOx
FavIconizer
fenkql7pbqog7AgiljAebjeAeqpgpyv
feuOiboqpfjupcobthfqlteqb9jviigkcwqpabp
ffplgXvgi dd3ycgrl 33hXno v cXwlmwwo
fIhaIjuqutIl q ldyntp jtxybasg
findtronic
forlt9dcd o b f kqgeec9evkk9br
fpRkuppwdlxucouovnny5rsqrkdxR5rpows
FqncvjvasykFomctjx ud4tdsd FywwkFw
fsF7ymmnksffreladrkrFvidvp
fvfcmQQmc7qimi7i Qg7gyfbsQm
g 3qgkwohjxyvrhyRRpblpy xpcubpx3cwejR
g8gsb8lvmufeikjyfbvs wii
gakkhsfomirity3rcaxhhprqsjpj iy
gkiYl br8xoqtr fwcnv
gmcskkkwpajaogcied5wvklltwkv sfoHgfg
gMtett0cl00acqabyasp
gnbhintmxgcyisut cyxixhKb
gnubieryyOlqndp tinhfrluhnsq9vvqO
go0ioKyrehyKKnwfbsKh ywqcw0yieonk dqne
gt gfnqw xjlNwibbgvkuusdrean
guypnkdn aqcrkciqulqsuwfq
gvojgauwc7odd fnlapwvr
gvoscpsxqrkoaaks8Sadoih
hkjoxgnxh sprokkxfqmmiype5omwos5sfrhifk
hphujhux eCds6jqd6ymCmfugjsu6dqu
hpqwksatkiuaasfeluxB rxpcqsBiitnB
hqbgmkbll1y1UqtrUUu1dqth
icaudpssdlwlrcinsh3ctcdifvvetyCrihpc
inobxmqbinbowqpodiqqqwyye5
inpwhdW irkhatwrqvdftptpWfpjveevkuj
ioFyceotjqkqryvkng fjs
iqaWyamxauu4lxdijl4i
IUS Spder Tch
jahysjcymawusia5dfgiqrfq
jevxQppwvifQvndtckw5ibkeykowhbdixjq
jhnic2du2wcxwoboumkg CgnChjxlho m
jixyUtqxUmnmempgboqwjUhcom
jmcyecg4xiltogh tbymdxwnxykgc wgdm
jnhh3pgmlxwuwjihhdpetmhauyn3rkeMkrha
jobBpwje bwvt3hwbhkxgdt
jqhjvRq ouyfdjqhpxirmjhg
jrdd4hpcgflfovgchntcywwKjmK
jreue ejybfads vMpcqnMq
jrjvicolklptbygpmyk fm tfvgjsnru
jtkvrimqnbraovypsodhrfwbhaeq8ykCfr nd8r
jup0sFuidksqxaFdrqxv
jvobxslvgtmmi 2aylvxxubuiprsdxinvqq2x
k8qvhDyufpDktDhgbycdbk
kjewu4qpg4frfgjj h fyq4hj batjwobEolwt
klfnjkalbnirbhvwihvexlpmlkfhje
kqohkipauigoivwhc b kjneqxbhopclpg
krmdg1ffryryyut1gDxdhw
kykapeky
l4cjamdkn4mhmdoonnwh4jktdpmkqau
lachesis
lbafW9rWlnqqdh jqa9mburxWjatf
lcogstmcpacm tsbipulloc6mn6
lcxpwfeisfysmlesWmtef
ldm duugksOuvlOqxoqOqbkmqft bo0k qqpqu
lDqqcoaacpaoqj4bDmvlitukeicaowiqphuh
lfjWtsnugojvwrhntmswwyhWsfgs
lj frrdnijpvmwn 3j3Hws3uuuml y
ljebafcgvnee lhmfndyb6qxyMgei6mac c
ljif0 y krjntjgnCdfgk
lmnmioovy1 rvtchwjjqqfxaqvfwimqq
mcc5legvhqxlvaIqnieqeaI
mdanafw0xfegump0fa0daenjrwdi
mdlq1jnywqjrdxvtsyiprurbvdr
mhtwhluohpcsfhlwaubraeiwlwcefewww
midpQ0vcuskQwprtqglqnxkwafqh lmiaaj
mkkrCkttakcavCvvjgkyan l2CyagfjjC
mqdqkhpjtsxt tnbwkggnei
mqf rusyjgbhtecuk2tfll
msgshgbh 3rbuQxbcj eacQvgmibgagd
msn1eqgnasg1fvavysfs
mtywqAki9xhwadryfnyasccqdi9baogwxc
mwsmy jejwf6jnrufpynQiQqvwbeQyfn vlnQk
mwuxnvvg sbslujecdyycj
n2wpnd spr qhrhgnxdrxcvxfle
nbrnflhndfncnyvbpiddmhrnkkudfowkyuwl m
nddxtjbsasrsjDigvomqvxqtajb8d
ngmjamovvqmiqobwm g osnb cfahtdi
nQ4idreinnueqicogjo uxqgiiqsugborw
nxghcfCqx7ohw jCmotkwrujyr7CCkky7k7sxf
nxvwwk2ieyjtgp2yutfhllbmsf2kxwjGsbi
nybairuiuyybgibnwkhvd6n6ay Pxtmdoxatd
o yhnmhxfekhagff f7rexu
ofkjbrgytcrjaj ikjtsqxqq
oiqeobmstxomqdtxmjbeuofwihruswtedqyo
omroaf4wk4rbnmEawtwqpvxttyyusEikwepsnp
opktk4voxrfxufiuqjakhsn
oppefe9bfo9qtufkucocby
orjxlipvy44ombvpnkmrmdmjwwrmpx
ornrwm ijubdJkosfpesajvengam
osirutdqChxldfxllqwciCb
otgonhawhgjbdwFgrvsb
ouigjihv4p otkvnsxabjvac4kigyaft sXue
owxwgediutrtiyb6fdnyfnoo 6fcoubvu6gf
p3aieYveqwYxqf u3xY3rtiraanvvt3pxmmw
p6crahlwwhjh6btpjrpahmq
pgaygtggrxqmtiq rx9ycumhccwXcay
phjledDvDpjrvemqoqllkti
phwox4 ejwafnosscxvao4jujaknticun
plovtn5knhsqxsqjhiCwfoqnei
pnfnfcuydnx x4gnmbgs
Ppkw 0Patuvejemolar
prdxPyf5eceqpeabupwdonrj
puyyue9a9sbqlvypyoOouyk9pellOdhOOdm
qdqiolvlftcbRs6ubatqpkqv
qdr np1qcnsvqjwrnuqiypbwl
qiaqurpduisq7i7uelgWusjgfqc cgddmw7
qkdkvqe1ma1kvdIwbI11yrmraajvl
qLama eij8 cqqswsx bmirLgybnrjbcmor
qmkuw 6awaoq f wnpqc uqrprboxy
qmncLbaiofncggcmk kyvf sxipq
qNmpbltwcwroNgwhypwcjhskdxbwsjj hemj
qskfycqqmeswlsdkuy9burqiewj
qutstqvaw5sqhtelYwn5jYwwhpyybdx
qwsyrdbxcdnk clfhxhtjnlublixbs
r4yynicWkcxjfjlWrve
rmsJyh4kd4qcaoo dbsaa
rptasirnvhvxg2vgoyqmd
rtqjgsll0simGwtlkorhjnjpks
saDqqvjej6ablanembdDjojmpbpcbcs
sagvefmb0afvbgtpsLajgbLvnbgesewictkhdj
sbqktnr hlorqmnqbkqbahxst e1ctRsgj n
sk0whqNchctucvbkh cwje
skbqgjnfnsge7kbght7jgy
spgwwwdagukhiigbpyya vmhjpurgywevhsqym
sqkoumyrfehtf npjochmtb5pexdiro mrkb
ssgaxrcmqewrkunleudewenNr
stltyglFyf fvvoxnjFjcfxmsqpmtpFvyntger
su2td cmjfdpirjhcaqwgw
suapjrgoqotcfnx qbhsyyOmrus
sueaqsfojYaqxxesddgnxpqevk2r aomehfl
suindnuppepeheyodbkeockuebVr4xykx Vlr
suwrbgp ptwntbbisixsu
syqxdrw iodeqqvjOqu eldetx slam
t2ljilmvk2mpax2l powfmjgxwlgmadn
taavt0qEheayprgdtEtcqxaukck
tanguydoturvoyatrddotfr4ncetelecomdotc0m
taolxhxfnbedxqhhkklrq
tbgaasMpxalk ror0kfvc
tbggKyfqv fhiKgslaKaKhllvpwa
tfccxbb3K tyaxkmlcskjlKqrrjpq
tiyayktnOonOgundcjsehOfh
tnqsyehntg tvteQhpleiyvmsx
tovo ombmbchnovopwccexQuwq uooinlec
tpuwwqwm 9 wdtftuwoppxpncxpwj
trlibymxn eajvxrlk2h pm2s
tu3uTtgntmnnmhnwTnlhgkmkflmedtwm3bw
tv6bymmtyitnyq6Ojocaqowl j6wmehi
twmimunsupfamgk gwwpb
ugsnxelashqwsAradkvexjapabklrv3jiaA
uji dkg9 o9pymwvcsrbsiq skm9ugs
uKmqrncgjoxrfvv ekdsspdugdpppdnj
ul pvpqsLqaoky8vrx fdkLhoLv
uLfdewtqluvcfoskxverpljvvw ouojpsywc
uOrggrpcc6oOgwaOpafnx6tbuhjnbtnkiO
urohceindbxdhycivc9xnvcsxlB ncfnoa
usn7orhgybqvh sexrtCu
uutoxvntchplrahOabcO2 stkOjywaOqco2q
uwbg1rwbijwywxywxvitqp1vu NsyrrtgxwvgN1
uWebmasterWorldaveh2iykwlilrmhwygaryk
v tqggqb4ebU4ovswixlemyglajeojmmrUa
vdw t1tcxSkerxltSwydlki wu
vg6a urwlckepyxcipxqepqvtiweisx6cc
vlg2yclmkahpmtf2bkmyspmombjr
vm pwmmeibfwuysublrxvvjpwpEe
vslpyexuBynqBqBf4ucxectwnlj
vxxdCvnfjufxfjrCcjb r
wacdqfkitbsibkglhfeqbifskmaXl
wffp bg 6gwqekmquslnlktqc
wjwnnnlicDe3l plgsstioihe3spqo sggtDc
wlqfjvevccorsvgvbwsaaKqmr
wmtDiDyknoee DkilDrypuqDotpxy kxv cdt
wnql aekrrf w fikqqo4xpqpednwslmirvy
wraoadnnqs eai4 q bo r c Mhbxxhofrhp44b
wrnxsmikhlDwgmfjexjtlvDefrxw
wta lvGt qrcqy nwuq anl
wumkQswaQwk2wqkhmaaqicgpslw
wxiaqe8s mxt8up hCp
wxkcrjtcnodl1l wfcophfibvbnqp 1c
x gasswoamaFdy2 qoFsxFyodnk
xafpbou gA gyxcseyh lggldevurAugpjxfcg
xc2x1f-1
xeyd3k3gfplahkhdoveysp3MsiMhoo i3kM
xiuRoa0gvsrsannxwwwsumg
xjxoaxim xvptlto7yivqnwpchoychscgobgu
xlb9Ycgoomtrgmu9oegsn
xmPevvmyuyiug5kpeqpppfrqxdir
xpans hs sBcpnaxlheislfbeBb
Xplore G 18
xQpfsustngfey smvmimi
xsakwitpcppnridwvghvtdt vgwr6
y lfhtysoofksr2CtxitfiCsftxhxCioxkgwwv
yCnnrwoCyfhikruhfuikeceehsCuf7uyfpksiyf
ykjkm fpc8pypxbivofxbjJs
ylryjwsdkn sgVqdsggkrajmnmimxhyjcr
ymtoqpfbyqebmqyqanxytjjbnhdm
yprCl1ncitd1qckufiinvbbsklib
ytnhhlwjaxarnebkackgsdql8jvYdjepo
Yutmkc99tcxqh99fsmpkynjns 9foY9
ywoqrpblqwuhysKfbfqvntgslolruaytbxp
--End of List --

[wilderness]

Ocean,
You must have "thumb twiddling time" ;)

Compiling a list of random UA's for reference!

There was a time when I added the IP's of these types of requests to denials, however relaizing quickly that the list would grow very fast, even that was discontinued.

Today, and unless they are actually crawling, I just watch 'em.

Course that was until I began using Jim's "tranqs" ;)

Don

[Ocean10000]

lol Wilderness

Not really thumb twiddling time, I have the list to test my own systems with to see if anything breaks where it shouldn't. I record most user-agents strings that cross my path so I can test them against my browser config/profile identification file which I maintain for Asp.net 1.x sites.

My last count I had well over 25,000 complete headers including user-agent strings which I test my stuff against. I tend to test and test some more, just to be sure just about everything can be handled properly.

Ocean.

[wilderness]

I record most user-agents strings that cross my path so I can test them against my browser

Ocean,
I digitize and archive data previously published from primarily widget periodicals.
To query my archives, the most useful tool that I've found id Copernic. The software has updates, however updates never just add the new, they frequently take some of the old away as well. Thu I hate to rock the boat.

In the past year, my widget data has grown by leaps and bounds, however in order not to corrupt my widget data, I've had to curtail activity in other areas.
The end result is that I'm not able to query my IP materials in a manner I once could. Or at least with any hope of expanding the data.

Don

[incrediBILL]

Nice list Ocean.

I'd upload my list of them but I don't think Brett has enough disk space to handle it.

[GaryK]

Owen suffers from Pack Rack Syndrome. ;)

I keep about a dozen of these random UAs in my database for testing. Generally speaking I keep track of them until I've seen them at least three times from the same IP. Then I firewall ban the IP Address. But I never save the UA. I'd be in the same situation Bill mentioned if I'd saved all those random UAs.

[wilderness]

I've had an interesting IP change as a result of one these being denied this today.
[Certain days on my sites are very slow, due to the patterns of widgets on those days; these vists in sucession on a slow day]

This page draws many of those randoms (haven't a clue why).

88.191.37.zzz - - [20/Jan/2007:07:05:49 -0800] "GET
[mydomain.com...] HTTP/1.0" 403 - "-" "yexsnasHnHupufweqaHccnre"
209.11.246.zzz - - [20/Jan/2007:07:12:49 -0800] "GET
[mydomain.com...] HTTP/1.0" 200 29584 "-" "Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0)"

[GaryK]

The vast majority of my garbage user agents come from Japan. Added: And they all go after just two photo-intensive sites.

[edited by: GaryK at 11:01 pm (utc) on Jan. 21, 2007]