Forum Moderators: open

Message Too Old, No Replies

Szukacz/1.5

Polish search engine spider

         

Finder

12:08 am on Oct 26, 2002 (gmt 0)

10+ Year Member



Looks like it's spidering DMOZ directory listings. I'm sure they were disappointed that my site has no Polish on it despite the domain name. :)

They have a robot info page [szukacz.pl] but don't ask me to translate!

<added> Yes, it checked robots.txt and obeyed it. </added>

bull

7:23 am on Oct 26, 2002 (gmt 0)

10+ Year Member



yes, i can absolutely confirm this. it was here too, first robots, then the odp listed directory.

info:
[szukacz.pl...]

carfac

6:46 pm on Oct 26, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi:

I have had this one before... and, if I remember right, the creator has posted on this board about the spider.

I have found it to be well mannered, and have not yet seen amy need to block it.

193.218.115.254 - - [25/Oct/2002:20:31:47 -0600] "GET /robots.txt HTTP/1.0" 200 4058 "-" "Szukacz/1.5 (robot; www.szukacz.pl/jakdzialarobot.html; info@szukacz.pl)"

Hit 4 other pages...

dave