Welcome to WebmasterWorld Guest from 3.81.29.226

Forum Moderators: martinibuster

Adsense ads, on noindex pages?

     
5:06 pm on Oct 28, 2019 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Aug 30, 2019
posts:145
votes: 28


Hello-

Is it a problem, to display adsense ads, on page which have the noindex tag?

And what about pages blocked at the level of the robots.txt ?

Will the adsense bot visit the page , or not ?
5:34 pm on Oct 28, 2019 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:26456
votes: 1076


Noindex should not be a problem as long as you've not blocked adsesebot from crawling the pages.
If you've blocked adsensebot from visiting the page then you will have a problem.
5:37 pm on Oct 28, 2019 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Aug 30, 2019
posts:145
votes: 28


Okay, thank you. And, if the page is blocked at the level of the robots.txt file, I guess I need to add an entry to allow the adsense bot? Or may be the adsense bot doesn't mind the robots.txt?
5:40 pm on Oct 28, 2019 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:26456
votes: 1076


adsensebot is meant to respect all protocols, so i'd let it in with permission for adsense sites.
5:43 pm on Oct 28, 2019 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Aug 30, 2019
posts:145
votes: 28


Okay, thank you!
6:10 pm on Oct 28, 2019 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Aug 30, 2019
posts:145
votes: 28


Me -again,

So let's say, I don't want pages in the folder "/some_folder/" to be indexed by search engines, BUT, still want to be able to display ads on these pages (as well as anywhere else). Should my robots.txt file looks like this?


User-agent: *
Disallow: /some_folder/

User-agent: Mediapartners-Google
Allow: /


I am still confused, if we should use "allow" in robots.txt file, or a "disallow:" with an empty string (nothing after the dots)
6:23 pm on Oct 28, 2019 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:4558
votes: 363


You can disallow bots and allow the AdSense bot as in your code above BUT if those pages are set to noindex AND Googlebot is not permitted to crawl, they won't know that those pages are set to noindex.
6:43 pm on Oct 28, 2019 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Aug 30, 2019
posts:145
votes: 28


Yes, sorry, I mixed two concepts in my questions.