Forum Moderators: open

Message Too Old, No Replies

Szukacz violating robots.txt

Polish search engine

         

Finder

9:06 am on Dec 12, 2002 (gmt 0)

10+ Year Member



Szukacz has always been very polite on my site, making sure to check the robots file frequently. Unfortunately a few minutes ago I caught them taking images from a protected directory.

I wrote them a polite email explaining the problem. In the meantime, into the ban bin they go:

Szukacz/1.5 (robot; www.szukacz.pl/jakdzialarobot.html; info@szukacz.pl)

<added>Sometimes the UA is just "Mozilla/3.01 (compatible;)" but from the same IP: 213.134.142.50
I also realized they've been doing this since the end of October, but I didn't notice because of the innocuous UA.</added>

Finder

7:07 pm on Dec 13, 2002 (gmt 0)

10+ Year Member



Update: I sent them some log fragments and they realized that their ISP was using a cache to download items they did not request -- namely images in a protected directory. They find this behavior unacceptable and have contacted their ISP to resolve the issue.

If you ever have problems with Szukacz don't hesitate to use the contact email in your logs. They are very responsive folks out there in Poland.