Make them stop!

any luck?

[luckynh]

There has been this a spider all over my site from www.linkguard.com

IP 195.217.246.100
UA HTTP/1.0 Mozilla/4.0+(compatible;+www.linkguard.com+LinkGuard+Classic+1.0;+Windows+NT)

Any one have any luck getting them to stop hitting my site?

Thanks
Lucky

[BoneHeadicus]

This [webmasterworld.com] helped me with a similar situation.

[luckynh]

thanks,
but i'm on WinNT and dont have .htaccess

But this did remind me that in IIS i can ban IP adds so I'll try that

Thanks

[luckynh]

Little more diggin and I found

Q. What is your robot policy?

A.
To disable or allow LinkGuard robots into your site, please use the user agent 'LinkGuard'.
[linkguard.com...]

[JonB]

this spider i s all over my site too. What is strange is that this spider requests over 50 times my index.htm page in one day. Why would any spider want to request/visit one page more than 50 times a day?

Jon

[Kimihia]

It is looking for broken links.

Usually someone goes to linkguard.com and requests they spider your site looking for dead links.

Yes it is ferociously fast.

[daveATclickthinking]

Hi luckynh,
You can still ban their IP on NT. Presume you are using IIS4/5.

Under the properties of the site select "Directory Security" then select "IP address and domain name restrictions".

Hope this helps.

Enjoy ... have a great day all.

Dave

[luckynh]

>You can still ban their IP on NT

I took a more conventions route an e-mailed then before blocking their IP.

05 February 2001
"..Our systems comply with the robots policy in force and, as such, can be prevented from scanning your site by implementing the appropriate robots exclusion parameters or components (see [web-support.csx.cam.ac.uk...] for more details on this). We will however exclude your domain from all scan jobs run by our systems so you do not have to follow the above suggestion.
This change will be implemented in our next production system's release, i.e. within a fortnight..."

07 February 2001

I have placed an entry into my robots.txt file for your robot
User-agent: LinkGuard
Disallow: /

13 February 2001

"..Apologies for this, as well as for our late reply.
We were quite sure that we had not left any room for this to happen, but we
are now triple-checking. We stopped the incriminated processes in the
meantime to make sure we don't scan your site, and your robots.txt setup
means it will not be scanned any more in the future..."

Haven't seen their bot since.