Welcome to WebmasterWorld Guest from 54.198.87.238

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Simulating bots/crawlers ?

     
9:14 am on Jul 13, 2009 (gmt 0)

5+ Year Member



I run a image heavy site, and Yahoo, Microsoft, Google image bots/crawlers visit me frequently. All 3 of them have indexed many thousand of my images.

I've been thinking about this and I want to replace the images I serve to the bots with watermarked images which include my domain name on the image.

This is simply added by using mod rewrite checking on the useragents extracted from my access log files. However how do I test this ? How can I browse my website with a specific useragent so I can see what a bot will see ?

Thanks in advance.

1:34 pm on Jul 13, 2009 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Use the Firefox Browser with the UA Plugin.
1:40 pm on Jul 13, 2009 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



If you are a webmasterworld supporter member - use the SimSpider to change the agent.

Aside from UA checkers, you can run proxies like Proxomitron to run local. (proxo is freeware and no longer supported)

 

Featured Threads

Hot Threads This Week

Hot Threads This Month