homepage Welcome to WebmasterWorld Guest from 54.226.0.225
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Hardware and OS Related Technologies / Smartphone, Wireless, and Mobile Technologies
Forum Library, Charter, Moderators: bakedjake

Smartphone, Wireless, and Mobile Technologies Forum

    
Mobile site design question
Managing scrapers and bots?
maximillianos




msg:4170975
 7:16 pm on Jul 15, 2010 (gmt 0)

Hello everyone,

Been working on my mobile site for a few days now. Getting close to rolling it out. I have some concerns about scrapers and such abusing my mobile site for quick access to my data/content.

Anyone have any experiences with this?

For now I am thinking of limiting my mobile scripts that page through my content to XX number of pages. Picking a number that most mobile users would never reach, but a bot could hit rather quickly. Then telling them to go to our non-mobile site for our full index of content.

I figured that might thwart the bot scrapers that stumble into my juicy mobile version of the site.

Obviously this would slow down any good bots from crawling the mobile site completely as well. I was not sure if I should even care about that given that I can submit a mobile sitemap to the major search engines anyway?

What do you all think?

Thanks!

 

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Hardware and OS Related Technologies / Smartphone, Wireless, and Mobile Technologies
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved