PycURL/7.15.

Whats up here?

[maccas]

I have about 50 of these in my log files, the ip address, referring site and the file they are requesting are all different.

59.9.*.* - - [15/Dec/2006:14:56:09 -0500] "GET /folder/ HTTP/1.1" 200 7497 "http://somewebsite.tld/" "PycURL/7.15.5"

[jdMorgan]

Looks like a cURL derivative -- maybe "Python CURL" from an Asian IP range, apparently.

At any rate, it's not a browser and not a search engine robot. That user-agent would be denied on any of my sites.

Jim

[maccas]

Yeah I will ban it, just curious as like I said the ip addresses are different (Asia, US and various Europe countries) and the referrer are from all different sites. I guess they are proxies and the referrer is spoofed.

[pixels]

I've banned the agent PycURL/7.15.5 using robot.txt but it is ignoring and keeps visiting my site messing up my logs. It uses different IP address as well so I can't ban it from htaccess either :<

How can I stop it from coming!?

=======================================

Host: 121.1.6.130
/links.php
Http Code: 200 Date: Dec 17 19:53:15 Http Version: HTTP/1.1 Size in Bytes: 8141
Referer: [oradvocacy.org...]
Agent: PycURL/7.15.5

/an.php
Http Code: 200 Date: Dec 17 19:54:08 Http Version: HTTP/1.1 Size in Bytes: 14515
Referer: [abi.co.uk...]
Agent: PycURL/7.15.5

/xmas.php
Http Code: 200 Date: Dec 17 19:54:15 Http Version: HTTP/1.1 Size in Bytes: 33349
Referer: [suedafrikainfo.ch...]
Agent: PycURL/7.15.5

/sitemap.php
Http Code: 200 Date: Dec 18 01:02:59 Http Version: HTTP/1.1 Size in Bytes: 6489
Referer: [query.no...]
Agent: PycURL/7.15.5

/contact.php
Http Code: 200 Date: Dec 18 01:03:27 Http Version: HTTP/1.1 Size in Bytes: 5467
Referer: [signs-marks.com...]
Agent: PycURL/7.15.5

/instructor.php
Http Code: 200 Date: Dec 18 01:04:10 Http Version: HTTP/1.1 Size in Bytes: 8423
Referer: [ceilingchina.com...]
Agent: PycURL/7.15.5

/form.html
Http Code: 200 Date: Dec 18 01:04:18 Http Version: HTTP/1.1 Size in Bytes: 7131
Referer: [rassvet.org...]
Agent: PycURL/7.15.5

/books.php
Http Code: 200 Date: Dec 18 01:04:46 Http Version: HTTP/1.1 Size in Bytes: 14811
Referer: [rassvet.org...]
Agent: PycURL/7.15.5

/books.php
Http Code: 200 Date: Dec 18 01:06:01 Http Version: HTTP/1.1 Size in Bytes: 22583
Referer: [welshjobs.com...]
Agent: PycURL/7.15.5
=======================================

[jdMorgan]

> so I can't ban it from htaccess either

You can certainly ban it by user-agent name in .htaccess, though...

Jim

[pixels]

OK thanks. I will put the user-agent in .htaccess file.
The following is what I've got. Would it work now?

<Files 403.shtml>
order allow,deny
allow from all
</Files>

deny from 71.136.*.*
deny from 11.111.*.*

SetEnvIfNoCase User-Agent "libwww-perl/" bad_bot
SetEnvIfNoCase User-Agent "PycURL/7.15" bad_bot
Order Allow,Deny
Allow from all
Deny from env=bad_bot

[mcordeiro]

Hi, I have alot of visits from UA PycURL/7.15.5 with different IP addresses. I used this code in my htaccess but I still see this UA in my logfiles.

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^pycURL [OR]
RewriteRule ^.* - [F,L]

Can any one suggest how i should change this to ban PycURL?
Thanks.

[jdMorgan]

You *must never* use [OR] on the last RewriteCond before a RewriteRule, as you show above. If you do this, the code won't work.

This should work better:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^PycURL/ [NC]
RewriteRule .* - [F]

The user-agent will still show up in your logs, but if the code is correct you will see that the server response code is 403-Forbidden.

Jim

[pixels]

Sorry, jpMorgan,
I am TOTALLY confused.
Are you saying that what I've got in the previous post is no good and should I change it to below?

<Files 403.shtml>
order allow,deny
allow from all
</Files>

deny from 71.136.*.*
deny from 11.111.*.*

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^libwww-perl/ [NC]
RewriteCond %{HTTP_USER_AGENT} ^PycURL/7.15 [NC]
RewriteRule .* - [F]

[leadegroot]

No, he said the FINAL entry must not have an OR on it. Think about it - theres nothing else to OR with!
Try:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^libwww-perl/ [OR]
RewriteCond %{HTTP_USER_AGENT} ^PycURL/7.15
RewriteRule .* - [F]

(err.... not sure if the final line should have the NC on it or not)

Remember - if in doubt, its quite easy to install the appropriate extension on firefox to change your agent and then test the htaccess yourself :)

[pixels]

Thanks for the FireFox tip leadegroot.

Could someone tell me if I need [NC] on the final line or not please?

RewriteCond %{HTTP_USER_AGENT} ^PycURL/7.15
or
RewriteCond %{HTTP_USER_AGENT} ^PycURL/7.15 [NC]

Many thanks.
Pixels

[pixels]

OK I've got it if anyone wants to no what [NC] means.
"No Case", meaning match regardless of being in upper or lower case letters. And you don't need [OR] either so it should be like the following.

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^libwww-perl/
RewriteCond %{HTTP_USER_AGENT} ^PycURL/7.15
RewriteRule .* - [F]

[jdMorgan]

You certainly do need [OR], because the user-agent cannot be both libwww-perl and PycURL at the same time, and that is what would be required to invoke the rule unless the implicit "AND" is overridden with [OR].

Unless you are sure that the user-agent case never changes, use [NC] on both RewriteConds, and [OR] on all but the last RewriteCond. More information is available in the Apache Forum Charter and in the Apache section of the WebmasterWorld library.

Jim

[pixels]

Thanks jpMorgan.
Merry Christmas to you all!

pixels