Welcome to WebmasterWorld Guest from 54.226.23.160

Forum Moderators: open

Message Too Old, No Replies

Firefox wants to get rid of your custom 404 error pages

Official plugin takes users to wayback machine instead

     
2:39 pm on Aug 4, 2016 (gmt 0)

Administrator from CA 

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 8, 2003
posts:3883
votes: 61


Mozilla's Test Pilot add-on is rather basic when it comes to that. It reacts only when a 404 not found error is recognized by the add-on. The add-on won't display the prompt if a site returns a different kind of error, or is redirected.

The notification reads: "This page appears to be missing. View a saved version courtesy of the Wayback Machine".

You may click on the link to open the Internet Archive website to read an archived snapshot of the page on the site. Please note that you may middle-click on the link to open the page in a new tab in the Firefox browser.

[ghacks.net...]
3:46 pm on Aug 4, 2016 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:25843
votes: 847


What! That is just stupid!
4:03 pm on Aug 4, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9047
votes: 752


404 exists for a reason. This time the boffins at Mozilla have sniffed too much cat nip!
4:15 pm on Aug 4, 2016 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:25843
votes: 847


I'm not sure it's cat nip, tangor. ;)

This is the sort of thing i'd expect on April 1.
11:03 pm on Aug 4, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15446
votes: 739


Uh.... Firefox doesn't know the difference between 404 and 410?

This is worrying.
11:04 pm on Aug 4, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


I have not seen this.
3:50 am on Aug 5, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9047
votes: 752


Firefox knows, it's the busy boffins trying to assert new frontiers that need no exploring. Sheesh. Just produce a good browser that is fast, secure and user configurable!

(We can dream, right?)
5:35 am on Aug 5, 2016 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:4161
votes: 262


I noticed that of all the new FF Experimental Add-Ons that one has the fewest "Test Pilots". I'd guess they signed on for the ride so they could give them some of that feedback they're asking for. What's next? A scrollbar that takes you to some random page in your browser's History? That could be fun.
6:18 am on Aug 5, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9047
votes: 752


Instead of things like this, I want the "Winning Lotto Ticket" add on. The one that pays off 100% of the time. Now THAT would be something actually useful. :)
6:45 am on Aug 5, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


So what happens if the page is not in the Wayback Machine?
6:59 am on Aug 5, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9047
votes: 752


So what happens if the page is not in the Wayback Machine?

Also makes me wonder since I have disallowed (and removed) some sites from wayback YEARS ago. (sites dependent upon Latest and Most Recent Info and the Old Stuff was NOT DESIRED)
8:08 am on Aug 5, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


Tangor... As do I (reason I asked) but according to that article, to my understanding, this add-on only attempts to forward the browser to the Wayback Machine if the page does not use code to redirect to a custom 404 page. So if properly redirected, this gizmo does not kick in and won't affect my sites.

However, surely this is browser hijacking and should be challenged and stopped soon enough?
1:02 pm on Aug 5, 2016 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1974
votes: 68


Pretty cool Idea.

Question is whether WayBackMachine's infrastructure is ready for it? What is next? "Ads On all over" someone else's dirty laundry?

Ye, pick on a small guy, go on.

Would You like to know More? << that came from a SyFy movie.
8:32 pm on Aug 5, 2016 (gmt 0)

Preferred Member from US 

10+ Year Member

joined:Mar 10, 2004
posts: 460
votes: 47


This is simply stupid. My sites return a special 404 page with links to major parts of the site, search box, etc. This would defeat the purpose of it, and have a poorer user experience.
3:52 am on Aug 6, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


My copy of Firefox just updated to 48.0 and says "up to date."

The title of this thread "Firefox wants to get rid of your custom 404 error pages" is a bit misleading IMO. Firefox 48.0 does not do this. The add-on in question, Mozilla's Test Pilot, is not default, in other words, it was not added when the browser updated.

So IMO this is a non-issue, especially if properly coded redirects to custom 404 pages are done either at the server level or the account level.

- - -

Further, when searching Firefox's Get More Add-ons for "Test Pilot" the Mozilla utility returned this message:
About this Add-on
This add-on has now been discontinued. The next update will automatically uninstall itself from user's profiles.
So voila... end of story
1:36 pm on Aug 6, 2016 (gmt 0)

Administrator from CA 

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 8, 2003
posts:3883
votes: 61


So IMO this is a non-issue, especially if properly coded redirects to custom 404 pages are done either at the server level or the account level.

The plugin, if installed, prompts the user to redirect to the Wayback Machine if the page is stored in their archive and your server returns a 404. No matter how you have your server configured, if you're returning a 404 status code (regardless of the HTML content you may provide back with it), the plugin prompts the user to redirect to the Wayback Machine.

Further, when searching Firefox's Get More Add-ons for "Test Pilot" the Mozilla utility returned this message:

You must be looking in the wrong place. Look here [testpilot.firefox.com]. Also you can see the source code [github.com] yourself.

The title of this thread "Firefox wants to get rid of your custom 404 error pages" is a bit misleading IMO

That this plugin has been given somewhat "official" test status indicates to me that they as an organization think the state of 404 errors are confusing for users, intend to do something about it, and are rolling out their solution to interested testers. Apologies if you were misled, but I think it's important to know about this stuff before it becomes default.

[edited by: bakedjake at 1:53 pm (utc) on Aug 6, 2016]

1:47 pm on Aug 6, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


Well I guess so. I couldn't find that add-on page. I stopped looking when Mozilla gave the discontinued message.

Just installed Firefox for Android directly from testpilot.firefox.com. That add-on either is not included or it doesn't work on the pages I tested.
2:08 pm on Aug 6, 2016 (gmt 0)

Administrator from CA 

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 8, 2003
posts:3883
votes: 61


properly coded redirects to custom 404 pages

A couple more notes:

404 is a status code returned by a web server to the client. It can be returned along with HTML content that is rendered by the browser. The plugin detects when this code is returned in the HTTP headers and prompts the user to redirect to the Wayback Machine if the file is found there. Even if you're using a custom 404 page, the prompt is displayed to the user.

The plugin prompt does not appear if you 301/302 redirect in the event of file not found (although you shouldn't be doing this anyway - best practice is to serve a 404 with content).

The prompt does not appear if you serve a 200 OK with HTML content in response to a file not found event. This is not best practice either but many webmasters misconfigure their custom error pages this way. ;-)

The prompt does not appear if you serve another type of code (like 410 Gone).

I have no idea what happens if you use a META refresh tag to handle your custom error page (whether returned with 200 or 404); I didn't test that.
2:26 pm on Aug 6, 2016 (gmt 0)

Administrator from CA 

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 8, 2003
posts:3883
votes: 61


Also, interestingly, IA keeps the clickthrough data server side and their disclosure of the information collected through the plugin indicates that they know the clickthrough rate - they show an example here [github.com].

I'm going to write to IA and ask if they'll give me an idea of CTR.
2:34 pm on Aug 6, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


Assuming the Mozilla discontinued message is in error and this feature goes forward, it would seem the only way to avoid having our traffic hijacked (or lured away) is to make sure the Wayback Machine does not keep archived copies of our pages.
2:54 pm on Aug 6, 2016 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:4161
votes: 262


When I followed the links in the article I landed on a Test Pilot page at Mozilla. It showed 3 different add ons that volunteers could sign up to "Test Pilot" and this 404 Add On had about 3,600 volunteers at that time vs. 200k - 400k for the other two (sorry did not write it all down). It did not look like anything to worry about but interesting to know what ideas are out there.
3:46 pm on Aug 6, 2016 (gmt 0)

Administrator from CA 

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 8, 2003
posts:3883
votes: 61


After having this installed for a bit I think this is probably great for users even though I hate it as a webmaster. The plugin was useful to me twice for non-webmaster reasons in the first hour I had it installed. I now want this as a default feature in my day-to-day browser.

I imagine these prompts see near 100% CTR, because a 404 page (even if it's beautiful, custom, with a splendid call to action or list of useful pages) is simply not that compelling compared to a link to the content I was actually looking for!
10:56 pm on Aug 6, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


My objection to the Wayback Machine & the so-called Internet Archive goes back many years. I could easily rant for hours about copyright infringement and their unethical tactics, but that's been done many times with these guys and they're still in operation.

When I see my favorite browser considering aiding and abetting by diverting my traffic to them, I once again feel they've somehow slipped under the door.

I do not feel this is great for users. Personally, if I take a page down, I do it for a reason. Other than bot exploit attempts, all 404s at my sites are typos by the user or poorly written links at remote sites. My custom 404 page answers that and give them a search-site tool to get to the correct page.

How long will it take the prompt to add "did you mean...? Here are some related pages from other sites at the Wayback Machine."
6:36 pm on Aug 7, 2016 (gmt 0)

Full Member from US 

10+ Year Member

joined:Apr 11, 2006
posts:243
votes: 21


Question is whether WayBackMachine's infrastructure is ready for it?


It would have been a good idea if implemented correctly from the start. It can look like your site on acid, and the wayer back you go the worse it gets.
1:40 am on Aug 8, 2016 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1974
votes: 68


Ha!

Nice!

Notwithstanding that there is an Army of scarpers out there, na? ;)
8:57 am on Aug 8, 2016 (gmt 0)

New User

joined:Nov 26, 2015
posts:1
votes: 0


Sounds quite a foolish thing to do on three fronts: CRO, UX and SEO.

Search engines expect us to 404 permanently removed/unavailable pages. They encourage us to return custom 404 content driven by data sniffed from the URL. I guess, so long as we start to return 410s then the plugin might not trigger....
If many FF users adopted this plugin, might search engines be more reluctant to land Firefox users on page levels of websites which they know to expire regularly? We might see a shift in entry points for FF users: screwing with CRO and UX.

Custom 404 pages can be a big win for CRO and UX.

And we have very strict bot-blockers, so wayback machine's not been able to crawl our sites for a few years.....so is this plugin going to have to pre-fetch the Wayback Machine page to find out whether there's an archived version of the page? I can't see this working very well TBH. Too many reasons why this is not a good idea IMHO.
11:39 am on Aug 8, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9047
votes: 752


Things should just be simple. If a page isn't there (for whatever reason) 404. If it is GONE, 410. Helpful idiots don't need to complicate things by trying to pretty up that side of the web.

And who selected wayback (archive.org) as the goto source? I suspect the index cache pages by the major search engines would be of better value, and more reasonably up to date (and more likely to N(OT have been blocked by the website for such historical records).
1:02 pm on Aug 8, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Feb 3, 2014
posts:1299
votes: 380


Why would they want top server up old outdated and likely incorrect information? Stooooopid
4:09 am on Aug 12, 2016 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:July 29, 2007
posts:1858
votes: 113


Seriously?

User-agent: ia_archiver
Disallow: /
That, when placed in my robots.txt, takes care of the wayback machine and

<meta name='googlebot' content='noarchive'>
takes care of Google's cache, which I noticed Yahoo has been serving up from Google lately. I've never benefited much from having archived copies of my site out there, in fact I feel it's cost me some visitors. But to go out of their way to force visitors not to see my custom error pages that have helpful links and such in them, game on... blocked.

If they start ignoring robots.txt and meta tags too I'll file DMCAs and block IPs etc... there is no need to be hiding custom error pages, imo. This is stupid imo, what's the payoff?

Webmasters have been losing control of their own sites enough as is.
4:39 am on Aug 12, 2016 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


Seriously?
User-agent: ia_archiver
Disallow: /
That, when placed in my robots.txt, takes care of the wayback machine

Not quite, sadly. IA has been documented to circumvent robots.txt and use other UAs to scrape content from web sites.

If you do not want your content published from their servers without your explicit permission, IMO it is necessary to use several methods concurrently:
disallow their UA via robots.txt
block their IP ranges from access to your server
block their UAs from access to your server

And keep a diligent watch to see if they are still getting your content; if so, use the legal processes designed to protect content owners from copyright infringement.
This 33 message thread spans 2 pages: 33
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members