Welcome to WebmasterWorld Guest from 18.208.159.25

Forum Moderators: Ocean10000

Message Too Old, No Replies

Pingdom

How to stop them

     
2:14 am on Jan 23, 2007 (gmt 0)

New User

10+ Year Member

joined:Jan 23, 2007
posts:7
votes: 0


Greetings, everyone! This is my very first post on WebmasterWorld.com. I have thought about joining WebmasterWorld.com for some time, but this present issue (post) was the fuel to push me to join. I am hoping that even jdMorgan himself will contribute to this thread. If the moderators feel that this thread needs to be moved, feel free. I am totally new to the structure here, but felt this was probably the best forum to put this.

Recently, I noticed something strange in my analytics and access logs, from a User-Agent called Pingdom. I went to their site and found out information about them. I'll try to encapsulate what I found.

(I would rather not link to their site and give them any link power. I hope no one else will either. Just type "pingdom" in Google and they come up #1. That should be good enough.)

When you first hit their site, you notice a high-color-contrast site. It looks striking / attractive.

Supposedly, their site services are designed so that you can truly know the up / downtime of your web-site. Sounds good, right?

Well, they also have what they call Pingdom GIGRIB. This is a service that they have, which accumulates data from thousands of sites. Apparently, users sign-up for an account/profile and begin submitting web-sites to be pinged by Pingdom. There are individuals who have submitted over 30,000 sites to Pingdom. Associated with their profile ID is a link to their own site.

The most disgusting thing that I find, in their advertising, is this statement, "Use the same tools to gain more knowledge about the infrastructure of your competitors and compare with your own."

I signed up for a free trial account, just to see what my "competitors" could possibly learn about my site(s). #1, The service allows people to ping whatever site they chose. That's just downright disgusting! #2, People can ping sites to test HTML pages, FTP, POP3, IMAP, etc. Again, that's just wrong!

I discussed this issue with my hosting company. They said, technically, anyone can ping these services. However, why would they do so? Why would any honest person do so?

Be that as it may, whether you agree with my analysis or not, I am desiring help in how to block them from all of my sites.

Apparently, some of these people who are setting up ID's for the Pingdom GIGRIB are using proxy servers. And, Pingdom uses many servers from every continent. Supposedly, you can just put "Pingdom GIGRIB" in your robots.txt and that will work for blocking that service. However, that service is not what your "competitors" will be using to ping your site.

Right now, because as I said, I signed up for their free service, their user agent for your "competitors" pinging your sites is "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)" and if need be, for the benefit of others, I will post the dedicated IP's they are using.

I would like a simple solution for permanently blocking them from my sites. I do not want my "competitors" pinging me or their Pingdom GIGRIB pinging me.

Is there a simple way to write an .htaccess file so that it would simply block any User-Agent that includes Pingdom?

I don't want this thread to primarily be about .htaccess, but I am wanting help in that area too. I know plenty about .htaccess, but not everything there is to know.

If I may, I would like to link to someone's blog post that I found, when searching on the keywords "user-agent pingdom":

TITLE: An Open Letter to Pingdom.com and JDCDesigns.com on the Topic of Referrer Spam

LINK: (removed)

I feel that this blog page will help to show someone else's frustrations that mirror my own.

Your thoughts? jdMorgan?

[edited by: volatilegx at 4:36 am (utc) on Jan. 24, 2007]
[edit reason] removed blog link [/edit]

4:49 am on Jan 24, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


Hi james96 and welcome to WebmasterWorld! I'm not sure how you can keep Pingdom from pinging your various services, except by blocking them at the firewall. Maybe Jim (jdmorgan) will have a better answer for you.
11:14 pm on Jan 24, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5507
votes: 5


james,
second the Welcome to Webmaster World.

However, why would they do so? Why would any honest person do so?

ping and tracert are just tools. Honesty may or may not be related to their use.
Internet providers have used both for years to time the accuracy of their services.

Both may be run from the command line (run) of Windows.
The earlier versions of Windows did not allow copy and pasting in the DOS screen (Command in XP), this availability in XP made the tools more useful.

4:12 am on Jan 25, 2007 (gmt 0)

Full Member

10+ Year Member

joined:May 5, 2003
posts:319
votes: 0


In your .htaccess file, place these directives, if your hosting company allows Mod_Rewrite Overrides.

Options +FollowSymLinks
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} pingdom [NC,OR]
RewriteCond %{HTTP_REFERER} pingdom [NC]
RewriteRule .* - [F]


These conditions state that if the User Agent, or the Referer field contains the word "pingdom," case insensitive, to deny access and return a HTTP 403 error

Wiz

[edited by: Wizcrafts at 4:13 am (utc) on Jan. 25, 2007]

1:05 pm on Jan 29, 2007 (gmt 0)

New User

10+ Year Member

joined:Jan 23, 2007
posts: 7
votes: 0


Thanks, everyone, for the welcome!

Wilderness, thanks for your reply. I know your points are valid, but Pingdom is not just doing ping and tracert in an unorganized, curious manner. They are organizing an effort to ping every site on the WWW. And, they are providing tools for people to use their services, I believe for malicious reasons. But, that's my opinion.

Wiz, thanks for your post! That's just what I was looking for! I will try it out and get back to you.

Thanks again for everyone's time! If anyone else wants to contribute, I would surely appreciate it!

(P.S., I see the link to "An Open Letter to Pingdom.com and JDCDesigns.com on the Topic of Referrer Spam" blog post was removed. That's fine. I didn't know if it would make it or not. But, I encourage you all to do a Google on that post and read it. It's rather enlightening.)

1:10 pm on Jan 29, 2007 (gmt 0)

New User

10+ Year Member

joined:Jan 23, 2007
posts: 7
votes: 0


By the way, Wiz, can you recommend a good resource for writing code in .htaccess? I have tried to do many searches on the internet for such code as you recommended, but can't seem to drag it out of the WWW. Maybe you have a particular web-site or book that you could recommend to me?
1:29 pm on Jan 29, 2007 (gmt 0)

New User

10+ Year Member

joined:Jan 23, 2007
posts: 7
votes: 0


By the way, Wiz, the reason I am so thankful for you posting that .htaccess code is because Pingdom is disregarding my robots.txt. They specifically put what user agent they use for their GIGRIB service (Pingdom GIGRIB) on their web-site and yet they disregarded my robots.txt to ping my main index page anyway.