Forum Moderators: LifeinAsia
My thinking is that if some of your content is generated server side depending on variables that aren't available to a scraper then the scraper will not get full funtionality if he / she piches your site. Maybe they steer clear of dynamic sites. Any thoughts?
One thought though, if your site is mainly .asp or any other server side technology, does that give your site some protection from scraping?
Scrapers see your site in exactly the same way legitimate bots do. Doesn't matter if you have .asp, .php or .html at the end of your filenames. Try surfing your site with JavaScript switched off to get a feel for how it will look to a bot.
Or put the other way round: the only content a bot is (currently) not able to see is content post-loaded into a page via AJAX-type technologies. Which would hinder the scrapers, but also not be available to the legitimate bots.