Forum Moderators: DixonJones

Message Too Old, No Replies

500 hits from Wget User Agent

Am I being copied?

         

affboy

1:38 pm on Jun 22, 2006 (gmt 0)

10+ Year Member



Hi There, im new to webmaster world.

I recently (2 weeks ago) put a site up live on the net. I have been monitoring my stats and I have had 500 hits in 2 weeks from a user agent claiming to be Wget. Im worried that someone is using this to copy my site and penalise me in the SERPS.

Can anyone give me some advice as to what to do? I have blocked it in my HTACCESS file but is this a good idea or am I just being paranoid?

Any help would be great,

Thanks

stapel

5:20 pm on Jun 22, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



WGet is a file downloader. Five hundred accesses with WGet would suggest that your site is being scrapes, at least to the extent of hundreds of your files.

You might want to post a "Terms of Use" page, including a specification that scraping the site is not an acceptable use. Then people can't complain when they get banned.

You might also want to look into installing a "bad bot" script, which can automatically ban IPs exhibiting suspicious behaviour.

Eliz.

mailorder

5:44 pm on Jun 22, 2006 (gmt 0)

10+ Year Member



Tricky one, it could be spiders?

affboy

5:57 pm on Jun 22, 2006 (gmt 0)

10+ Year Member



I only have 3 pages on the site.. 500 times sounds a bit extreme and i only have 3 pictures on it.
Do you know anywhere where i can get or see someone else terms of service page? i have no idea where to start.

Thanks,
Affboy

stapel

6:49 pm on Jun 22, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Please check your "sticky mail" messages.