Is it a problem, to display adsense ads, on page which have the noindex tag?
And what about pages blocked at the level of the robots.txt ?
Will the adsense bot visit the page , or not ?
engine
5:34 pm on Oct 28, 2019 (gmt 0)
Noindex should not be a problem as long as you've not blocked adsesebot from crawling the pages. If you've blocked adsensebot from visiting the page then you will have a problem.
JorgeV
5:37 pm on Oct 28, 2019 (gmt 0)
Okay, thank you. And, if the page is blocked at the level of the robots.txt file, I guess I need to add an entry to allow the adsense bot? Or may be the adsense bot doesn't mind the robots.txt?
engine
5:40 pm on Oct 28, 2019 (gmt 0)
adsensebot is meant to respect all protocols, so i'd let it in with permission for adsense sites.
JorgeV
5:43 pm on Oct 28, 2019 (gmt 0)
Okay, thank you!
JorgeV
6:10 pm on Oct 28, 2019 (gmt 0)
Me -again,
So let's say, I don't want pages in the folder "/some_folder/" to be indexed by search engines, BUT, still want to be able to display ads on these pages (as well as anywhere else). Should my robots.txt file looks like this?
User-agent: * Disallow: /some_folder/
User-agent: Mediapartners-Google Allow: /
I am still confused, if we should use "allow" in robots.txt file, or a "disallow:" with an empty string (nothing after the dots)
not2easy
6:23 pm on Oct 28, 2019 (gmt 0)
You can disallow bots and allow the AdSense bot as in your code above BUT if those pages are set to noindex AND Googlebot is not permitted to crawl, they won't know that those pages are set to noindex.
