joined:Dec 20, 2004
Been working on my mobile site for a few days now. Getting close to rolling it out. I have some concerns about scrapers and such abusing my mobile site for quick access to my data/content.
Anyone have any experiences with this?
For now I am thinking of limiting my mobile scripts that page through my content to XX number of pages. Picking a number that most mobile users would never reach, but a bot could hit rather quickly. Then telling them to go to our non-mobile site for our full index of content.
I figured that might thwart the bot scrapers that stumble into my juicy mobile version of the site.
Obviously this would slow down any good bots from crawling the mobile site completely as well. I was not sure if I should even care about that given that I can submit a mobile sitemap to the major search engines anyway?
What do you all think?