Welcome to WebmasterWorld Guest from 107.20.5.156

Message Too Old, No Replies

Great spambot trap - but requires a 1x1 pixel image link

Will this hurt?

     

bluemi

1:57 am on Nov 14, 2006 (gmt 0)

10+ Year Member



I found a really cool and easy to install spambot trap - works like a charm. Only problem is that it requires a 1x1 pixel image linking to a "forbidden" directory ("good" robots are excluded from it with a robots.txt file so that only "bad" bots will follow the link and then get blocked from the site). Any ideas if such a link will hurt my site?

[edited by: tedster at 2:00 am (utc) on Nov. 14, 2006]

kamikaze Optimizer

2:13 am on Nov 14, 2006 (gmt 0)

10+ Year Member



I use it and it seems to work well, it does not hurt you at all.

Just remember to add the deny to the robots.txt a few days before adding the remainder of the code on your site. Some bots only check the robots file every few days.

bluemi

2:22 am on Nov 14, 2006 (gmt 0)

10+ Year Member



Great, thanks, that's good to know. I'm still running it in "test mode" on a minor site, so no worries about the deny in the robots.txt. Looks like a great tool though.

jpavery

2:36 am on Nov 14, 2006 (gmt 0)

10+ Year Member



the smart scrappers obey the robots.txt file.

I did the same on my site (just collected IPs) and I got a lot of IPs.

I got concerned "pre-link fetchers" might be fetching the banned link and thus adding a regular surfer to my blocked list.

Paul

MThiessen

3:25 am on Nov 14, 2006 (gmt 0)

10+ Year Member



1x1 dots are common "spacers" dreamweaver uses them, as well as other editors. 1x1 images are ultra common and not likely to cause problems.

tedster

7:47 am on Nov 14, 2006 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Just to clarify, it's not the 1x1 gif itself that's a concern here, but the fact that it's linked.

bluemi

6:06 am on Nov 16, 2006 (gmt 0)

10+ Year Member



Well, it could have been nice but I just removed it again after one of Google's bots got "trapped" (65.222.176.125). I wasn't aware that Google doesn't obey the directives in the robots.txt...

bluemi

6:17 am on Nov 16, 2006 (gmt 0)

10+ Year Member



Oops, sorry, that was of course not Google but a real spambot... mixed up the lines ;-)

ann

8:34 am on Nov 16, 2006 (gmt 0)

WebmasterWorld Senior Member ann is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Wouldn't that qualify as a hidden link and if discovered could get you penalized by Google?

Ann

MThiessen

7:24 pm on Nov 16, 2006 (gmt 0)

10+ Year Member



"Just to clarify, it's not the 1x1 gif itself that's a concern here, but the fact that it's linked."

Ahhh ok, now that is a hidden link... Don't know what I was thinking. 1x1 spacers are common but never used as a link.

bluemi

7:27 pm on Nov 16, 2006 (gmt 0)

10+ Year Member



Hi Ann. Yes, that's what kind of worries me too. On the other hand, when I think of all these sites with hidden texts and links that are ranking extremely well in Google I feel that the Googlebots' ability to detect spam are ridiculously overestimated anyway. But then, you never know...

ann

10:40 pm on Nov 16, 2006 (gmt 0)

WebmasterWorld Senior Member ann is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Yep,

Shot myself in the foot with that one a long time ago. Used it to link to a site page that I was working on and wanted no one to see just yet. Kerplunk! Bottom of the serps I went :)

Ann

bouncybunny

11:40 pm on Nov 16, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Why not add it as a regular text link at the bottom of your site, with the text "spambots follow this"? ;-)

The chances of scraper bot authors actually looking at your site are pretty remote and the bots are illiterate.

ionchannels

12:52 am on Nov 17, 2006 (gmt 0)

10+ Year Member



I do this, with text at the bottom of my page which says:
Do NOT click here (access will be revoked)
If someone or something clicks, their ip address is put into my firewall, but before that I explain why they have been banned and provide my email if they are a person who clicked the link "accidentally". I usually get 2 or 3 people a week who click it out of curiosity. I collect about 10 spambots every day with it.

nippi

6:04 am on Nov 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



ionchannels.

i am curious about this. I don;t have a static ip, and if ran a scraper, and you banned me, you'd not in fact ban me anyway, jsut the next person using that ip.

Seems pointless, you didn't stop me in the first place, you then didn't ban me, jsut some other poor sap.

jdMorgan

6:40 am on Nov 17, 2006 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



{quote} Wouldn't that qualify as a hidden link and if discovered could get you penalized by Google? [/quote]

Not if the link target is disallowed by robots.txt. That makes it fairly obvious to GoogleBot that you didn't add the link for tricky ranking purposes.

Linked 1x1 gifs with the link target disallowed by robots.txt are one of the common methods for tracking and logging page accesses. You'll often hear them referred to as "Web beacons" in either Web or e-mails contexts.

I've been running this script (or one like it) since the latter part of 2002. The site it's been on longest hasn't dropped out of the top 3 positions in the SERPs at any time before or since.

Jim

bluemi

6:58 am on Nov 17, 2006 (gmt 0)

10+ Year Member



Thanks Jim, that's very useful information!

bouncybunny

8:36 pm on Nov 17, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Out of interest, is there an easy way for a non-techie to log visitors IP addresses when they visit a specific forbidden directory/page.

AWStats isn't that specific.

Thanks