psbot/0.1 Anyone else getting hit by it?

picsearch bot

[Alphawolf]

psbot/0.1 [picsearch.com]

It was on my site for a while, but I disallow my images directory. Without access to my raw log files, DeepMatrix can't tell me what it looked for.

Does this bot obey robots.txt?

TIA

AW

[edited by: Alphawolf at 10:20 pm (utc) on Mar. 14, 2003]

[jdMorgan]

Alphawolf,

I have observed that psbot obeys robots.txt on my site.

Jim

[Alphawolf]

OK- wonder what it is spending 6-10 minutes doing then if I don't let it look at my images directory.

AW

[MarieC]

Alphawolf, it obeys the robots.txt on my site too. Do you have this:

User-agent: psbot
Disallow: /

or something else? I'm no expert, but it occurs to me that if you are allowing it or all robots on your site, but disallowing a certain image directory, it may very well be staying out of that image directory, but looking through your other pages. Hence, the 6 - 10 minutes? I don't know. Anyway, the bot page of picsearch seems reasonable and they do have an e-mail and contact page for problem reporting.

[Alphawolf]

Hi MarieC,

I have this:

User-agent: psbot/0.1
Disallow: /

Should I remove the /0.1?

Just added that recently...

But I have had the following for a long time:

User-agent: *
Disallow: /images

Among other directories. Is it often the case that these sort of bots would need to be specified by user agent in the text file?

TIA

AW

[jdMorgan]

I use just "psbot" without the version number. The robot name used in robots.txt is typically not the same as a complete user-agent string as seen in raw access logs. Version numbers are rarely - if ever - specified in robots.txt.

Jim