homepage Welcome to WebmasterWorld Guest from 54.161.155.142
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Stopping Voila from indexing
How to do it
experienced




msg:4104284
 8:04 am on Mar 25, 2010 (gmt 0)

Hello,

There is one crawler Voila who is eating my bandwidth on daily or alternate days basis. I have already blocked the same in robot file ong time ago but still it is coming again n again, my stats shows

Voila 1832+89 207.73 MB 24 Mar 2010 - 10:12,

i have checked for voila correct agent name if i am using wrong name to block but i did not find anything.

Is there anybody who can help me stoping this voila, is it vola.fr or something else..

 

dstiles




msg:4104829
 10:40 pm on Mar 25, 2010 (gmt 0)

For the past few months I've been seeing the UA:

Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com)

I allow it on the simple string voilabot, which should do for blocking as well.

February there were 23,000 hits across a server of several dozen web sites. That's from a total of 145,000 hits from all permitted bots including google, yahoo, msn (the highest by far), vagabondo, yandex, ask, twiceler and a few others. It's about 12th on one of the more active sites (google is 4th).

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved