Interesting from Yahoo!

[volatilegx]

Date: 04/10/2006, 04:45:29
IP: 202.43.217.59
Host: web9.search.cnb.yahoo.com
UA: curl/7.10.7 (i386-portbld-freebsd4.3) libcurl/7.10.7 OpenSSL/0.9.6g zlib/1.1.4

[Pfui]

I haven't seen that but I wonder if they're (& I hope they're not) using curl, a.k.a. cURL, the way one hyped site is -- to scrape 'n' save entire pages, not links, for members' private viewing at any time, blatantly violating "NOARCHIVE" (& robots.txt), etc. Here's the other site's UA:

"curl/7.13.1 (platform-here) libcurl/7.13.1 OpenSSL/0.9.7i zlib/1.2.3"

Similar, eh? Dang.

(Aside: Am being obscure about the scrape 'n' save site on purpose, sorry, because it took some serious sleuthing to ID that UA as being from there -- they use two; one to hit, one to scrape -- and I don't want them to change things any time soon. They were also surprisingly rude in e-mail when I simply asked about what they were doing. The tech guy who e-mailed me was hostile right out of the gate and went downhill from there.)

Oh, yeah. I block curl because of its (ab)use as a speed-downloader browser helper.

[Staffa]

A couple of times I have seen Yahoo being blocked on my site for no apparent reason.

Well, curl could be a very good reason, it's banned ;o)