homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 42 message thread spans 2 pages: 42 ( [1] 2 > >     
mediapartner bot and opera browser
does it hurt to block this spider?

 4:07 am on Jun 26, 2004 (gmt 0)

I am wondering the following.

When people browse a webpage with Opera browser google mediapartner will visit that page just a couple minutes later.

If there is a second visitor that also use Opera the user will see your page but it will also see adwords in the advertisement box from opera.

This is really bothering since the ads are very good on topic and are from competitors.

I strongly believe this is not the way of fair business. If I want to have advertisements on my page I place them myself and I get paid for them.

The fact is that people want to use a free browser and that the webmasters of the pages they visit have to pay for their pleasure of using opera browser.

OK...my questions:

Does it hurt to block the mediapartner spider?. I mean will your google ranking drop or no regular googlebot visits anymore.

Anyone experience with this? By blocking only the mediapartner bot. Does it solve the issue of displaying advertisements in Opera?

What is the robots.txt entry you have to add to ban this bot?



 8:56 pm on Jun 26, 2004 (gmt 0)

You mean AdSense is running on pages that aren't signed up with the program? Something doesn't sound just right there. That defitely doesn't sound like anything Google would do.


 9:18 pm on Jun 26, 2004 (gmt 0)


What I mean is OPERA browser.

If you use the free download version (which most people are using) the right corner of the browser shows a adbox with 2 adword/adsense advertisements that are on topic with the page you are viewing.

It is built in with the OPERA browser. Start OPERA yourself and you will see the box in the right top corner.

I strongly think this is not a decent way of doing business.

I do understand that advertisements are a way for OPERA to make money and of course they need money to survive. Competition is good. Don't take me wrong.

However they give the browser away for free in exchange for advertisements. OK...still nothing wrong with that.

But who are being the victim of this business way?. Exactly. The webmasters and business that have people visiting their homepage by using the OPERA browser.

Not only because users will click on the advertisements in the OPERA BOX. that lead them to the competition.

But also because they will get a lower clickthrough rate on their own advertisements (esspecially adsense). Since the OPERA advertisements will be the same but more official/dominated placed.

It is common sense that many business don't want to show advertisements of their competition on their homepage.

Think about HP that is running an advertisement for a DELL computer. Do you think this will ever happen?...but with the OPERA browser it is possible that DELL places a advertisement in the OPERA built in box that is running adwords/adsense.


 9:24 pm on Jun 26, 2004 (gmt 0)

OK, I see what you're saying. I do use Opera and have seen those ads. What's happening is that when the Opera browser hits the site they're running the bot to get the context of the page so that the ads can be targeted to that page, that's what you're seeing then?

Banning mediapartner wouldn't work if the site itself is running AdSense.


 10:02 pm on Jun 26, 2004 (gmt 0)

Does it hurt to block the mediapartner spider?. I mean will your google ranking drop or no regular googlebot visits anymore.

the mediapartner bot and the regular googlebot appear to be different entities running from different IP ranges. In my experience mediapartner spidering has had no positive effect on googlebot / SERPs etc. (otherwise every SEO and his/her pet of choice would be surfing like crazy with Opera ;-), so presumably the reverse is true (banned mediabot won't negatively impact the regular bot).

Anyone experience with this? By blocking only the mediapartner bot. Does it solve the issue of displaying advertisements in Opera?

As a member of the mediapartners-bot fanclub I have no idea ;-). This link might be of interest:

What is the robots.txt entry you have to add to ban this bot?

This page might be of use to you:



 2:56 am on Jun 27, 2004 (gmt 0)

If you are running adsense on your pages and someone visits using Opera, do they still see adsense? Wouldn't this be the same as having two ad units on one page?

As an aside, you can use .htaccess to block those using opera, or redirect them to the firefox download page.;)


 3:08 am on Jun 27, 2004 (gmt 0)

If people are surfing other similar sites they'll be seeing those same ads anyway, and whichever way you look at it the competition is only a click away for anyone to go to the search engines and look further.

I've got adsense running right on my pages - definitely the competition, some of it pretty stiff. I'm still getting bombarded with more inquiries than I could possibly handle, and my site's far from being a mover and a shaker compared to what I'm surrounded by in the SERPs.

We can look at it this way - if they're surfing using Opera with the ads, they're seeing those ads all the time. If they're still surfing they're still looking, so an actual page still has a major advantage over a simple two line ad that a lot of people learn to ignore anyway.


 4:02 am on Jun 27, 2004 (gmt 0)

Powdork....funny suggestion to lead them to a download page. But if you do that the user will immediately be gone and you lost your opportunity.

Also the other suggestion .htaccess

This doesn't work since you can modify the user agent in opera and many people do that because of banking software requirements. Therefore they will have a UA that say MSIE6

I think I go for the blocking mediapartner approach in my robots.txt

I think that is the best way to go for now.

As remark I hear noone about if they think it is a good thing or a bad thing that opera does this.....what do others think about those ads in opera?


 4:20 am on Jun 27, 2004 (gmt 0)

I far prefer the Adsense - and there is an option to choose.

That Adsense doesn't run on the web page at all, it runs in a little window at the top of the Opera browser and it's barely noticeable. There's still only one Adsense group on the page itself.


 5:24 am on Jun 27, 2004 (gmt 0)

It was just a joke. I wouldn't block opera users ever. But speaking of folks I do block. What happens if you use Opera's free browser while using ad blocking software like Norton's personal firewall (which I wholeheartedly condemn)?
Does your machine explode?;)


 9:45 am on Jun 27, 2004 (gmt 0)

I have no experience with blocking the Mediabot but it sounds like it shouldn't be a problem. Most likely the only consequense will be that Opera users will be served less relevant ads - which is by no means a guarantee they won't click on them anyway.

Blocking or redirecting Opera users is useless: that's throwing away a potential customer.


 12:14 pm on Jul 22, 2004 (gmt 0)

Just found out that blocking the Mediabot doesn't help.

I now see the Mediapartners bot trying to visit pages behind a password on a site which disallows all robots. Doesn't even bother to read robots.txt.


 6:25 am on Jul 23, 2004 (gmt 0)

If the page is listed in Google I am pretty sure that you will see targetted(competitors) ads in the Opera Browser ad space.

Mediabot only tends (not always) to visit pages it has no information on (ie not in the Google index.) - so blocking mediabot may not have the desired effect.


 12:29 pm on Jul 23, 2004 (gmt 0)

yes...found out that as well....however it can be coincidence but my ranking dropped for all my keywords slowly the last couple weeks since I added the ban on the mediabot.

I didn't see mediabot in my log files anymore but I removed the entry in my robots.txt today since it doesn't matter as indicated that the target ads are still there even on new pages (added in the last week). The ads will show up as soon as the page is in the normal google serps.


 1:38 pm on Jul 23, 2004 (gmt 0)

> what do others think about those ads in opera?

Got no problems with ads, since they are pretty small. And I wouldn't worry about them at all.

I do think this is a sound business strategy. If you are afraid of those little ads, you should build better content or improve your products or both.


 4:25 pm on Jul 23, 2004 (gmt 0)

I've had the Google ads showing in opera since they started doing them and I have never clicked on even one ad. I simply don't notice them any more.

If people do click the ads, however, you don't lose the visitor as the new page opens in a new tab and not over the original page (as per normal adsense)...


 12:49 am on Jul 24, 2004 (gmt 0)

Powdork pondered, amost a month ago...

> What happens if you use Opera's free browser while using ad blocking software like Norton's personal firewall (which I wholeheartedly condemn)? Does your machine explode?;)

I had advertising.com (who serves Opera their ads) blocked at the firewall long before I installed Opera - I've never seen an ad in Opera. (Took me a bit to figure why that was...)

No explosions, but my monitor does smoke a bit... :)

(Powdork, is it Norton you condemn?)


 4:09 am on Jul 24, 2004 (gmt 0)

The behavior of the ad based Opera is no worse that using the Google toolbar in IE with the PR lookup turned on. As long as the user is aware they the url they are visiting is being reported to a third party, I see nothing wrong with it.

They just MUST be made aware that their url browsing is being reported.


 4:31 am on Jul 24, 2004 (gmt 0)

As an aside, you can use .htaccess to block those using opera, or redirect them to the firefox download page.;)

I would redirect them to IE instead, I want my customers to have the best. :)

Back to the topic I do thing it's bad and think that's why they never could last in the browser market. Sure they have been around for sometime, but a lot of people won't download it because of that advertisement like myself.


 5:09 am on Jul 24, 2004 (gmt 0)

Opera is doing nothing wrong, whatsoever. If you find it to be a huge problem then ban opera users and redirect them to a webpage telling them why they wont be able to access your site and supply links to where they can download some alternate browsers you would prefer surfers use to access your site.

Problem solved.


 3:39 pm on Jul 24, 2004 (gmt 0)

Why not cloak for mediabot and give them a page full of promotional text about FireFox. Wouldn't that cause them to put FireFox ads up?

troels nybo nielsen

 6:15 pm on Jul 24, 2004 (gmt 0)

I find it difficult to believe that this is a significant problem for anybody from a purely practical point of view. It is possible that there is a loss of revenue because of those ads though I guess it would be difficult to verify, but how much do Opera users weigh in your statistics?

The ethical aspect may be different, though. I personally do not think so, but I understand those webmasters who do not like Opera's attitude.


 10:31 pm on Jul 24, 2004 (gmt 0)

"advertising.com (who serves Opera their ads) blocked at the firewall"

Or for those who don't have a firewall an alternate way would be to edit your local copy of hosts file as follows:


Then advertising.com will not resolve. Maybe if we had a good running list going of these it could be helpful? I have done a similiar thing many times over for: oscar.aol.com when need be with the desired results. ;)


 1:21 am on Jul 25, 2004 (gmt 0)

I've only just stumbled upon this thread.

Some time back [actually early Nov, 2003]it was heatedly debated over at Amazon Associates, over on the Opera Forums and I think here as well.

I don't know if I'm allowed to post those links here.

One of our WebmasterWorld participants developed a script so that "unregistered" users are re-directed to the Opera registration page.

The theory was that any Opera Browser showing competing Ads was denied access to your site.


 2:28 am on Jul 25, 2004 (gmt 0)

Isn't this a copyright issue? Can you sue Opera for lost revenue they are making off your copyrighted material.


 2:51 am on Jul 25, 2004 (gmt 0)

You've heard from the webmasters ... now hear from an opera fan ...

The fact is that people want to use a free browser and that the webmasters of the pages they visit have to pay for their pleasure of using opera browser.

Exactly. You got that right.

Why should I (or even opera) be concerned about your business? Opera is looking to make money (like you), and I am looking for a good browsing experience ...

As a potential customer, I find the ads useful BECAUSE they provide me more options - the competitors (who perhaps are better). In fact, I have even mailed a suggestion to opera to incorporate a 'show similiar sites' button (powered preferably by Google) and display that in the sidebar pane. Does that mean you will also ban Google Bot or not show your website to opera users?

Rather than assuming that you are indeed losing business, why not use this particular problem to your advantage and do something more constructive ...

--> Use the business that you get from Opera users as a MEASURE of success of your current business model because it can be assumed that he/she might also have visited your competitors, but chose YOU.

--> Advertise using Google AdWords and send some jitters to your competitor too ... :)
[ Personally, my perception of credibility of an online front increases if they advertise. Why, because that means they are willing to invest money (a rarity in online business now a days, what with everyone trying to make a quick buck). Again, its only one of the small things ... ]

--> (Going a bit off the path) Heck, if you are confident about your business service, even place a link on your site to a Google similiar site search that displays your competitors.
(I consider this a bit unethical, but since ethics depends on a persons perception ...) : Create a page on your site listing your competetitors. Choose the competitors that you belive offer an inferior service compared to yours.

[Do you know that DMOZ ( [dmoz.org...] ) has a category that contains links to all the criticism and spews against it : [snipurl.com...] ]

Ultimately, it all comes down to this - if you have a sound business model you don't have to worry. A smart customer WILL always find your competitor. Period.

Typos and errors regretted. Flamers don't bother ...


 11:12 am on Jul 25, 2004 (gmt 0)

sfxmystica sounds a bit too much like a Claria/Gator rep.


 12:54 pm on Jul 25, 2004 (gmt 0)

Opera doesn't touch the ads on the page at all, and it is the ad on the page which i in the best position. You don't even notice the Google ads in Opera after a while.

I doubt you're losing a lot of money because of this.

And comparing it to spyware or programs that replace ads on pages with their own is a bit out there, isn't it?


 7:42 pm on Jul 25, 2004 (gmt 0)

I'd like to take issue with the description of this thread on the main page of Webmaster World.

here are some very interesting issues that come up from the usage of the free version of Opera. It is a version that includes ads by Google. Those ads are contextual. Hence, there is very little real world difference between Opera and a AdWare/Scumware/Malware laced version of IE.

There is a world of difference. IE is bad because of ActiveX, which can install programs and viruses and who knows what. Also IE is built-in to Windows, leading to a pandora's box of security issues (some of which have yet to be patched). Opera is not in the same league at all. It is a standalone program that does not run ActiveX, so is considered to be much, much safer than IE. The free text ads in no way allow scumware to enter Opera. They are just ads of text and links.

I find the quote as it stands now will give people who have never tried Opera the wrong impression. We should be encouraging people to switch away from the plagued IE. Please consider rewording the quote.


 9:45 am on Jul 26, 2004 (gmt 0)

Sometimes it helps to exaggerate a little to bring everything into focus : (Imagine) a webmaster on seeing this topic saying -

"Oh God, what if Google adword has picked up my urls on forums that use Adword and is displaying my competitors ad ... damn the forum, damn the forum admin, damn google adword, damn my competitors, damn the internet ... yeah, I think I'll just shut down my business or scare people about softwares or sites with google ad or any other contextual ad that displays my competitor ads ..."

This 42 message thread spans 2 pages: 42 ( [1] 2 > >
Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved