Welcome to WebmasterWorld Guest from 54.162.224.176

Forum Moderators: Ocean10000 & incrediBILL & phranque

Apple Touch Icon 404s - Typical Solutions Don't Work

     
4:09 pm on Jun 11, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


I am having this crazy issue with those apple-touch-icon.png files. Not only are my logs getting filled with 404s, there are an inordinate amount of them (10x as many requests for them than I have traffic)

This happened a year ago and at that time I used htaccess to block all requests, and that solved the problem. Then I moved to this new host which uses the same stack (Apache/PHP) and no matter what I try, tons of 404s.

What used to work to block these requests was:

# BLOCK APPLE apple-touch-icon REQUESTS
<FilesMatch "apple-touch-icon-precomposed\.png|apple-touch-icon\.png|apple-touch-icon-120x120-precomposed\.png|apple-touch-icon-120x120\.png|apple-touch-icon-152x152-precomposed\.png|apple-touch-icon-152x152\.png">
Order allow,deny
Deny from all
</FilesMatch>


Then I 301'd all the variants to the core apple-touch-icon.png file. That has just resulted in getting even more 404s for that file:

# For APPLE apple-touch-icon REQUESTS
RedirectMatch 301 (?i)/apple(-touch-icon-.*)?.png /apple-touch-icon.png


When I try to direct navigate to any of the variants (apple-touch-icon-120x120.png etc), they all redirect in the browser to the apple-touch-icon.png, so why that is getting a 404 is a mystery. My host's support cannot figure this out either.

Then, I broke down and made these icons and put them in the root directory. Not surprisingly, even that didn't stop the 404s. All the other files in the root directory (robots.txt etc) are there and can be directly navigated to w/o any 404s. It's just these Apple icons that cause these 404 issues. Even when I did the redirect to the main apple-touch-icon.png, browser requests did redirect--but the 404s still piled up.

So far what I've tried is not working, even though all the other rules in the htaccess work.

Does anyone have any ideas what this could be? (I think it is a malicious bot given the amount of requests for these file proportionate to the actual sessions in Analytics)
4:46 pm on June 11, 2018 (gmt 0)

Preferred Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 361
votes: 33


The best solution I have is to create these icons with the right sizes and names they request. Once they get a clear 200 they do go away. I think these apple products are looking for icons and they repeatedly try to find them. Once the graphics are created with the right name they find it once and only once.

They are not bots but people, and are not malicious.
5:02 pm on June 11, 2018 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:3772
votes: 210


The icons are used as bookmarks. Different sizes for different devices. It's a good way to get your logo seen and it is easily shared to help build traffic. There are a lot of tutorials out there on how to make your own at no cost.
5:20 pm on June 11, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


The thing is that I did exactly this - made the various icons at different sizes and then still got 404s days later.

Another strange thing is that I have quite a few websites and none have these icons in the root dir or specified in the <head> and none of those sites get 404s.
9:07 pm on June 11, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14803
votes: 640


<FilesMatch "apple-touch-icon-precomposed\.png|apple-touch-icon\.png|apple-touch-icon-120x120-precomposed\.png|apple-touch-icon-120x120\.png|apple-touch-icon-152x152-precomposed\.png|apple-touch-icon-152x152\.png">

Er, all you needed to say was
<FilesMatch "apple-touch-icon[^.]">

Why does the topic header say “soft 404"? If anything, you’re creating a soft 404 by redirecting all those requests. Don’t bother; legitimate human UAs will try a few standard filenames and either get one they like or stop asking. There is nothing wrong with a 404 if you genuinely don’t have the file. You are not obligated to create something just because people keep asking for it. After all, you didn't make a /wp-admin/ directory just to appease visiting robots did you?
6:13 pm on June 12, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


I tried this again. Made the requisite icons and put them in the root dir, added the link in the head section and still I see tons of 404 errors.

How could this be happening when the files are there (so shouldn't trigger 404) and all the other files like robots.txt which are in the same directory can be navigated to in the browser. Even when I try to see these apple-touch-icon files in the browser, I can.

I agree, there is nothing wrong with a 404 if one doesn't genuinely have the file, but when I do and can see it myself in the browser, but still get 404's, this is weird.
8:15 pm on June 12, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14803
votes: 640


Are your newly created icons in the same place as your original apple-touch-icon (the one you were redirecting everyone to)? Is it possible there is some server-level aliasing and/or rewriting, so the requests are all looking in the wrong place and it's out of your power to change it? Also check your error logs and make sure that each 404 you find in access logs corresponds to exactly one "File does not exist" utterance in the error log, and that the given filepath (error logs show physical paths, not URLs) is what you expect it to be.

Option B, if you don't want to keep making new formats--I think there are about twenty of them--is to rewrite all of them to the form you do have:

RewriteRule ^apple-touch-icon[^.] /apple-touch-icon.png [L]

To the best of my knowledge, G### has not yet started flinging around Duplicate Content accusations when the file in question is an apple-touch-icon ;)
9:22 pm on June 12, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11366
votes: 148


I would look for clues in the server error log file.
9:58 pm on June 12, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


Thanks for that rewrite rule!

What I see in the logs is

85.xxx.xxx.xx - - [12/Jun/2018:07:09:39 -0500] "GET /apple-touch-icon-precomposed.png HTTP/1.1" 404 9397 "-" "MobileSafari/604.1 CFNetwork/897.15 Darwin/17.5.0"
10:40 pm on June 12, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14803
votes: 640


What I see in the logs is
Yes. Now pull the error log for the same time period--they should be saved in the same place as server access logs, though most people rarely look at them--and see what it says for this request. You should find the IP and the timestamp; the rest of the error log gives slightly different information.

:: shuffling papers ::

On a 404, the error log entry will look something like this:
[Mon Jun 11 18:27:50 2018] [error] [client 65.154.cc.dd] File does not exist: /full-physical-filepath-here/images/m-main417.png

<tangent>
Well, that was interesting. I knew the bingbot had an obsession with asking for lower-case filenames, but because I normally disregard non-200 image requests, I didn't realize they also did it with non-page files. (The image in question is really called M-main-etcetera.) Now I have learned something.
</tangent>
2:35 am on June 14, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


There doesn't seem to be anything in the error logs (provided in cPanel on this shared host).

What is strange today is that all the apple-touch-icon variants are redirecting to the basic file, apple-touch-icon.png, but now that file, which is in the root dir, is triggering more 404 requests than ever.

There was a similar issue I read about here some years ago. Someone suggested a badly coded bot could be causing this. (thread was "apple-touch-icon.png 404s?" from 4:27 pm on May 27, 2014).
2:46 am on June 14, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11366
votes: 148


this is the thread ArchStanton referenced:
[webmasterworld.com...]

is there a pattern to the requests that get a 404 or all all such requests getting a 404?
have you examined the responses and server error and access log files when you have directly request those resources - both the redirected urls and the base icon file itself?
3:24 am on June 14, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


I do see a pattern. All of the variant sizes (eg /apple-touch-icon-120x120.png) throw a 404. But the base icon (apple-touch-icon.png) Throws a 301.

The same thing, even from my IP:

99.xxx.xx.xxx - - [13/Jun/2018:22:04:39 -0500] "GET /apple-touch-icon.png HTTP/1.1" 301 257 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36 OPR/53.0.2907.68"
4:10 am on June 14, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14803
votes: 640


What's the next request, immediately after the 301? (Reminder: a 301 response tells the visitor to make a new request, so if it's a human there should certainly be a follow-up.) You said “even from my IP”. You mean when you’re trying it out for yourself? What do you eventually see on your screen?

Unfortunately it is not easy to log what someone was redirected to. Possibly if I knew more than two words of php I would know how to go about it.

:: wait, stop, rewind ::

I have only this instant noted that in your initial post, you had a mod_alias redirect:
RedirectMatch 301 (?i)/apple(-touch-icon-.*)?.png /apple-touch-icon.png
Do you use mod_rewrite at all? (It's pretty hard not to.) If so, you need to change all mod_alias redirects (Redirect by that name) to mod_rewrite syntax, and put them in the appropriate order.

What does (?i) mean--is that mod_alias syntax for [NC]? (You can tell I have not used this mod very often in the last 5+ years.) I'm a little confused about what requests the rule is supposed to work on. Do people come in asking for "apple.png" ? They certainly wouldn’t ask for "apple-touch-icon-.png" but that is what the .* part allows.

No matter what mod is issuing the redirect, the target should include the full protocol-plus-hostname.
4:45 am on June 14, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


I got that from the mod_alias/script_alias syntax. From what I understood, (?i) matches URL path. I got rid of this yesterday and used your example with mod_rewrite:

#Apple touch:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^apple-touch-icon[^.] /apple-touch-icon.png [L]
</IfModule>

The next request after the 301 (which was me) is for another IP for another file that was 200'd. When I tried to request the file "apple-touch-icon.png" I see it in my browser, but saw it as 301 in the log. (that is really weird, no?)
5:37 am on June 14, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:8349
votes: 642


BTW, ArchStanton, welcome to Webmasterworld, and thanks for the stimulation!

Aside: 404s are not a bad thing, and are also very easy to strip out of logs. Just asking why....
5:56 am on June 14, 2018 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:3772
votes: 210


The next request after the 301 (which was me) is for another IP for another file that was 200'd. When I tried to request the file "apple-touch-icon.png" I see it in my browser, but saw it as 301 in the log. (that is really weird, no?)

Your canonicalization rewrite would come after the apple-touch-icon rule, so no it is not weird, if you requested with a different protocol than your canonicalization rule provides. In other words, if your request was for https://example.com/apple-touch-icon.png and you're rewriting that to https://www.example.com/apple-touch-icon.png then you would expect to see the 301 response.

Now if you send all visitors to https://www.example.com and requested the icon as https://www.example.com/apple-touch-icon.png then, yes it is weird.

6:50 am on June 14, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11366
votes: 148


The next request after the 301 (which was me) is for another IP for another file that was 200'd. When I tried to request the file "apple-touch-icon.png" I see it in my browser, but saw it as 301 in the log. (that is really weird, no?)

look further down in your server access log (after the 301 from your IP) and you should find a subsequent 200 response logged from your IP address.
what does that show?
2:10 am on June 15, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


BTW, ArchStanton, welcome to Webmasterworld, and thanks for the stimulation!

Aside: 404s are not a bad thing, and are also very easy to strip out of logs. Just asking why....


Thanks! I had account here ten years ago and jdMorgan and incrediBILL really helped me out back then.

My issue with this I guess is me getting lost in minutiae. and also worry that this just leads to extra resources on the shared host.
2:11 am on June 15, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


Your canonicalization rewrite would come after the apple-touch-icon rule, so no it is not weird, if you requested with a different protocol than your canonicalization rule provides. In other words, if your request was for https://example.com/apple-touch-icon.png and you're rewriting that to https://www.example.com/apple-touch-icon.png then you would expect to see the 301 response.

Now if you send all visitors to https://www.example.com and requested the icon as https://www.example.com/apple-touch-icon.png then, yes it is weird.


I am using the non-www rewrite consistently.
2:23 am on June 15, 2018 (gmt 0)

New User

joined:June 11, 2018
posts: 10
votes: 0


look further down in your server access log (after the 301 from your IP) and you should find a subsequent 200 response logged from your IP address.
what does that show?


When I paste [mysite.com...] what I see in the logs is just one entry:

99.xxx.xx.xxx - - [14/Jun/2018:21:12:05 -0500] "GET /apple-touch-icon.png HTTP/1.1" 301 257 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36 OPR/53.0.2907.68"


All the other IPs logged have requests for the icon variants which show 404 and one each showing the base icon (apple-touch-icon.png) showing 301. Since mine is direct navigation it only shows one for that specific file - the 301.
2:43 am on June 15, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14803
votes: 640


Now, wait, that's weird. A human visitor--for present purposes, a webmaster is considered human--receives a 301 response and they don’t make a follow-up request?

Is there something wrong with the redirect target? If you’ve misspelled the hostname, they'll end up on someone else's site, and the request obviously won't show up in your own logs. If the content of the 301 is seriously malformed, it's possible the browser would not send in a request at all.

Did we establish earlier that you don't have access to your site's error logs?
12:41 pm on June 15, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11366
votes: 148


use your browser's web developer tools or a header checker to see the response chain when you request an icon variant.
carefully inspect the Location response header the should be provided with the 301 response.
there should be subsequent request for that resource.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members