Welcome to WebmasterWorld Guest from 54.196.217.43

Message Too Old, No Replies

Great spambot trap - but requires a 1x1 pixel image link

Will this hurt?

     
1:57 am on Nov 14, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:May 8, 2002
posts:179
votes: 0


I found a really cool and easy to install spambot trap - works like a charm. Only problem is that it requires a 1x1 pixel image linking to a "forbidden" directory ("good" robots are excluded from it with a robots.txt file so that only "bad" bots will follow the link and then get blocked from the site). Any ideas if such a link will hurt my site?

[edited by: tedster at 2:00 am (utc) on Nov. 14, 2006]

2:13 am on Nov 14, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:July 25, 2003
posts: 608
votes: 0


I use it and it seems to work well, it does not hurt you at all.

Just remember to add the deny to the robots.txt a few days before adding the remainder of the code on your site. Some bots only check the robots file every few days.

2:22 am on Nov 14, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:May 8, 2002
posts: 179
votes: 0


Great, thanks, that's good to know. I'm still running it in "test mode" on a minor site, so no worries about the deny in the robots.txt. Looks like a great tool though.
2:36 am on Nov 14, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 30, 2002
posts:128
votes: 0


the smart scrappers obey the robots.txt file.

I did the same on my site (just collected IPs) and I got a lot of IPs.

I got concerned "pre-link fetchers" might be fetching the banned link and thus adding a regular surfer to my blocked list.

Paul

3:25 am on Nov 14, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 10, 2002
posts:531
votes: 0


1x1 dots are common "spacers" dreamweaver uses them, as well as other editors. 1x1 images are ultra common and not likely to cause problems.
7:47 am on Nov 14, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Just to clarify, it's not the 1x1 gif itself that's a concern here, but the fact that it's linked.
6:06 am on Nov 16, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:May 8, 2002
posts: 179
votes: 0


Well, it could have been nice but I just removed it again after one of Google's bots got "trapped" (65.222.176.125). I wasn't aware that Google doesn't obey the directives in the robots.txt...
6:17 am on Nov 16, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:May 8, 2002
posts: 179
votes: 0


Oops, sorry, that was of course not Google but a real spambot... mixed up the lines ;-)

ann

8:34 am on Nov 16, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 25, 2002
posts:2605
votes: 0


Wouldn't that qualify as a hidden link and if discovered could get you penalized by Google?

Ann

7:24 pm on Nov 16, 2006 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 10, 2002
posts:531
votes: 0


"Just to clarify, it's not the 1x1 gif itself that's a concern here, but the fact that it's linked."

Ahhh ok, now that is a hidden link... Don't know what I was thinking. 1x1 spacers are common but never used as a link.

7:27 pm on Nov 16, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:May 8, 2002
posts: 179
votes: 0


Hi Ann. Yes, that's what kind of worries me too. On the other hand, when I think of all these sites with hidden texts and links that are ranking extremely well in Google I feel that the Googlebots' ability to detect spam are ridiculously overestimated anyway. But then, you never know...

ann

10:40 pm on Nov 16, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 25, 2002
posts:2605
votes: 0


Yep,

Shot myself in the foot with that one a long time ago. Used it to link to a site page that I was working on and wanted no one to see just yet. Kerplunk! Bottom of the serps I went :)

Ann

11:40 pm on Nov 16, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 10, 2006
posts:661
votes: 0


Why not add it as a regular text link at the bottom of your site, with the text "spambots follow this"? ;-)

The chances of scraper bot authors actually looking at your site are pretty remote and the bots are illiterate.

12:52 am on Nov 17, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 22, 2005
posts:95
votes: 0


I do this, with text at the bottom of my page which says:
Do NOT click here (access will be revoked)
If someone or something clicks, their ip address is put into my firewall, but before that I explain why they have been banned and provide my email if they are a person who clicked the link "accidentally". I usually get 2 or 3 people a week who click it out of curiosity. I collect about 10 spambots every day with it.
6:04 am on Nov 17, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 3, 2003
posts:805
votes: 0


ionchannels.

i am curious about this. I don;t have a static ip, and if ran a scraper, and you banned me, you'd not in fact ban me anyway, jsut the next person using that ip.

Seems pointless, you didn't stop me in the first place, you then didn't ban me, jsut some other poor sap.

6:40 am on Nov 17, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


{quote} Wouldn't that qualify as a hidden link and if discovered could get you penalized by Google? [/quote]

Not if the link target is disallowed by robots.txt. That makes it fairly obvious to GoogleBot that you didn't add the link for tricky ranking purposes.

Linked 1x1 gifs with the link target disallowed by robots.txt are one of the common methods for tracking and logging page accesses. You'll often hear them referred to as "Web beacons" in either Web or e-mails contexts.

I've been running this script (or one like it) since the latter part of 2002. The site it's been on longest hasn't dropped out of the top 3 positions in the SERPs at any time before or since.

Jim

6:58 am on Nov 17, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:May 8, 2002
posts: 179
votes: 0


Thanks Jim, that's very useful information!
8:36 pm on Nov 17, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Mar 10, 2006
posts:661
votes: 0


Out of interest, is there an easy way for a non-techie to log visitors IP addresses when they visit a specific forbidden directory/page.

AWStats isn't that specific.

Thanks