Forum Moderators: open
I'm asking because I'm banning 216.239.39.5 (agent is UP.Browser/6.1.0.1.140 (Google CHTML Proxy/1.0)
), as this thing continues accessing directories disallowed in robots.txt.
I don't want to ban the googlebot.
The specific ip is mentioned, and seem to be used either for normal proxy or wap proxy.
There is list of most ip's that Google has on different sites, try doing a search on it. (spiderhunter is quite popular)
The reason this user-agent doesn't obey robots.txt is because it is not a robot. You are banning mobile users who use Google's HTML-to-CHTML translator!
Google also has an HTML-to-WAP translator. Check out the third thread in lazerzubb's list.
Jim
This thing keeps triggering my bad bot trap. I might modify the script or simply ban cell phone users - my site is a huge photo gallery and all images on my site are 800x600. There is no way they can be viewed on a WAP screen.
Anyway, am I right in assuming that banning 216.239.39.5 will not worsen my Google ranking?
But why would you want to ban the cellphone users? They can't be using a lot bandwidth.
