|Is AdSense causing 404 errors?|
im trying to narrow down a problem that i've noticed in the last couple of days, where a visit to a particular page on my site will result in a 404 redirect.
the page loads okay, but after a second or two it will redirect to a 404
Obviously that is not something that i can fix at my end, so i was wondering if anyone else has noticed this in the last few days?
its the 336x280 box or 300x250 box that im having problems with, and it happens about 50% of the time. so maybe its because of a particular type of ad, like an image one?
...i've set up a custom 404 file in my .htaccess file (which i've had on there for years), and i've just noticed that if i comment that line out, the problem disappears too.
presumably the file still cannot be found, but it no longer redirects to my custom 404 file. why would that be?
I've seen something similar with custom error pages, if not set up correctly. Like in pointing to virtual path not allowed, or not found. Removing the config pointing to the custom page then seem to be "fixing" the problem. In the logs it shows up kind of funny.
Are you sure the old custom page was working overall? If you asked for a bad page, you actually saw that custom 404 page?
I have seen similar problems with 403 pages, where I needed to add special directory permission in Apache to make it work. Specifically, in that case, I had filtering that would issue a "Deny" on seeing certain UAs. But rather than showing the custom 403 page, that internal redirect request would of course get a 403 as well, but now on the 403-custom (same UA, my mistake). :)
I had to put conditionals around access to the custom page, to prevent the error page from getting blocked too.
So in your case, could the 404 error in this case actually be because some other error is happening, and Apache on its way to show a custom error page instead hits a 404 because it cannot access the path for the 404 custom within that virtual host? A 404 on the 404-custom?
The log entries can get a bit confusing when those internal redirects happen. At times the error code shown in the log actually refer to something not expected.
[edited by: DeeCee at 6:28 pm (utc) on Mar 22, 2012]
if i put the line back into the htaccess file then the redirect works as planned.
i can write in a nonsense URL and i get my custom 404 page.
if i take the line out, then i just get the normal generic 404 page. so i'm pretty sure its working okay.
this is what i've got in the htaccess file... i dont have anything else around it.
ErrorDocument 404 /docs/404.php
You mention the redirect happens "after a second or two". Could it be a host or call that is timing out, host mis-typed, ..
You call it a redirect. Is your browser redirecting the page, or is the 404 code just showing in the server log? If the browser redirects, you might want to look at the HTTP headers passed around?
its the browser that's redirecting after a second or two. but it's definitely responding to something that AdSense fetches, because that's the only thing that makes the error disappear after i turn it off.
i've tried turning off everything else on the page one-by-one, including by own JS files, images and ads, and its AdSense.
ive tried looking at the headers for the 404, but there are no clues there. it just shows my original page.
...actually... does adsense pre-fetch stuff from the sites listed in the ads? maybe it is one particular site in the ad that's the problem, rather than adsense itself.
No, the ad-show does not call on the sites doing the advertising.
When I mentioned checking headers, I was thinking of using such as the LiveHTTP add-on for Firefox to see which headers and requests are being sent back and forth. Not so much the individual page's headers.
LiveHTTP should be able to show you specifically what is being requested and the timing of it all.
The fact that it is ending up doing a 404 on your server tells you that it is asking for something on your server. Otherwise the 404 error would land somewhere else.
the only 404 that comes up is from adsense. it is trying to fetch a very long URL which starts with
...but it's trying to fetch it from my domain. so thats why its redirecting to a 404 on my site -- it thinks that i'm the host
but the referrer for that URL is
do you know of a way to ignore a directory in htaccess without sending a 404?
Different issue. You are having an actual ad issue. You don't want to ignore dartiframe.html. (DART Iframe)
That is apparently something needed if your site (that problem page only maybe) uses iframes and ads.
If you are not a DoubleClick direct customer, it would seem to be a bug from Google, if they are serving badly formatted ads from double click.
From DoubleClick/Google help site:
Set the path to DART Iframe.html
If your site uses iframes, you must download the DARTIframe.html file to successfully run Rich Media ads in your website. (See Downloading the DARTIframe.html File for more details). DoubleClick recommends that you put this file in a directory named doubleclick at the root directory of your web server.
If you can't store the DARTIframe.html file in the recommended directory, use an attribute in an ad tag to specify the directory where this file is stored on your web server. This attribute is mtfIFPath=value, where value is the path to the directory (from the root of the web server) where the DARTIframe.html file is stored.
For example, store the DARTIframe.html file here:
To specify the path to this file using the mtfIFPath attribute, edit an ad tag like this:
The brackets on "[//]" is mine. Not from actual page.
perfect, thanks for your help with this deecee
just to mention one last thing... maybe its my imagination, but i seem to be getting a better variety of image ads on my site now that i've fixed this.
i have an events section, for example, and a lot of the ads are now about the actual event, rather than just the general subject.
maybe if you dont set this up, a lot of the doubleclick ads are blocked from appearing.
...might be worth trying, if you use image ads
Good to hear.
Yeah, it would be blocked on pages with Iframes.
Can I ask, what are you doing with IFrames?
What exactly is supposed to be in the /doubleclick/DARTIframe.html file.
i dont use iframes myself, but i've got one of those social media 'addthis' buttons on there which generates one
the file just seems to generate a
<script> block, validates the parameters in the url, and reports any errors back to doubleclick. its not very big.
Shoot.. I did not think of that. I have those as well in some areas. I have never seen a call-back for that file, though. I guess I'll have to keep an extra eye out for it.
Strange that Google/Adsense would now have pointed that out to people or fixed it yet. That has the potential to hit many sites they display ads on.
I take that back.. It is all over Google Groups.
Old problem BTW, See this webmasterworld link: [webmasterworld.com...] from 2008.