Xenu Link Sleuth 1.2b - Why would someone use this on one of my sites?

I know roughly what it is.

[diddlydazz]

Anyone had this before ?

Thanks

Dazz

[rogerd]

Lots of reasons, diddly. Maybe they want to get a map of your site, to see everything you have that's visible on the web? It's a lot easier to use Xenu than clicking on links. Plus, Xenu will follow hidden links and perhaps expose some content that you don't want everyone to see. Or, perhaps they want to harvest your outbound links?

I don't know if Xenu obeys robots.txt, but I doubt it. If being spidered by Xenu bothers you, using user-agent exclusion will discourage the casual snooper.

[diddlydazz]

user-agent exclusion ?

excuse my ignorance but do you mean in the robots.txt ?

Sorry about the question :)

Dazz

[rogerd]

No need to be sorry for asking, DD! Robots.txt would be the first line of defense, but assuming that fails, you would have to take one of several approaches: cloaking, modifying the .htaccess file, using ASP, etc. There has been some good discussion in the past here about dealing with unwanted user agents. If I can locate a relevant thread, I'll post a link.

[diddlydazz]

Thanks

whatever it was it only got one page is this normal ?

I thought it would have been back for more but it hasnt.

Dazz