Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Help with Google reverse/forward DNS - in layman's terms

         

ladyviola

4:10 pm on Mar 7, 2009 (gmt 0)

10+ Year Member



Hi! I have recently discovered that a few china based website are serving up an exact duplicate of my entire website; a redirect of some sort except when I look at the address bar--its their domain. I tried to fix this using a redirect of my own in .htaccess, but I feel this is a bandaid, and not a real solution. I have read countless threads pertaining to proxy hijacks and the like, but I am not understanding how to implement the reverse forward dns lookup to make sure only the good bots are visiting my site. Can someone give me instructions in layman's terms? Thanks in advance!

tedster

12:15 am on Mar 8, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Just in case you didn't catch our threads about this process - here are two:
[webmasterworld.com...]
[webmasterworld.com...]

When a request comes to your server and says that it is from googlebot, your server takes this 2-step approach in deciding how to respond:

1. REVERSE LOOK-UP
Using the IP address that request, your server does a reverse DNS lookup. Because Googlebots have reverse DNS enabled, the result should be a subdomain of googlebot.com. If it isn't, then your server does not return the content -- it's someone spoofing the googlebot user agent and your server can return a 403 Forbidden status.

2. FORWARD LOOK-UP
To make sure that the reverse-DNS entry isn't fake, your server now uses the full googlebot subdomain that was reported in step 1 and does a forward DNS look up. That should return the same IP address that the original request. Again, if it isn't, your server doesn't return the content.

In the second link I gave, incrediBILL offers some other practical tips for employing this method - be sure to check them out.