Usability Test Your Store Checkout

[engine]

I found a supplier of a product, and the site looked legitimate, so I decided to order a product.

Their product information seemed comprehensive and I hit the add to cart button. There was a continue shopping option, so I looked for anything else that might be of interest. Not this time, so I headed for the checkout.

I looked for shipping costs, and they were not readilly available, as with many stores. I thought i'd get that as I progressed through, so I continued. Eventually, I found a shipping cost description, which was a little woolly, imho. It did not state specifically how much it would cost for my region. It simply stated the cost for shipping to the store's local region. What use is that? Again, I thought it may come up at the next phase.

Being a new customer, there was an option to open an account. I duly competed the form with all * section filled in. I hit submit, and it came back asking me for a username and password. Thinking i'd missed it, the form had already refreshed with the error clearly described in red. I studied the form, and nowhere did it have an option to complete a username and password.

I tried it one more time (not sure why, but I did) and the same error came up.

It's at that point I decided it would be easier to try elsewhere.

I ended up going to amazon for a similar product.

I'm prepared to support the smaller etailers, but, really, they have to get their procedures tested for all eventualities. I wouldn't call this a wildly strange eventuality; a new customer!

Have you road-tested your store?

[piatkow]

Have you road-tested your store?

I once had two emails in quick succession, one saying how simple the site was to use and the other complaining that it didn't work. It was tested by two people but both were IT professionals. The moral is to get a non IT people to do the user testing (different operating systems and browsers of course) and to have somebody looking over their shoulder to record exactly what they do - but without helping.

[rocknbil]

I have limited resources, unlike many of you. But I do my best, and it seems to work.

Using an un-public copy, run it through "as is" in all browsers, pointed at the test gateway servers, so long as the test gateway servers accept data *exactly* the same as the live ones. Repeat in every browser.

Take every page state, view source, use the direct input of W3C validator to validate every.single.page.

Turn Javascript (and Flash, if it's involved, which it rarely is) off. Re-test.

Turn images off. Re-test.

Find as many of the least Internet savvy people I can find (averages between 3 and 7.) Buy them lunch in trade for a test. I go to their location, sit quietly and have them test it twice - once to let them see that I'm not judging their Internet skills. The second time they are more comfortable and the test is more accurate. I like vocal and opinionated people for this, I find those that are old and cantankerous are best. :-)

This is the best part of it, it always comes back with things I thought were clear can be better clarified. Apply changes.

(You're all going to hate this one) I have an old Mac G3 here I have left at OS 9 with IE5 for Mac and NN4. I call these my "acid tests." Yes, there is not a lot of sense in supporting these, but if it survives these, it will survive just about anything. Repeat the first test, JS on, JS off, in both.

Apply changes to the live site, repeat the first test live.

Throughout we have links to contact in the event of user problems, the most often is on the shipping summary page ("your shipping is too high!") When we changed the verbiage to indicate the prices are retrieved real-time from USPS, and a link to the USPS shipping calculator, these stopped.

The last reveals that your road testing . . . is never really complete, nor is your site.

I looked for shipping costs, and they were not readilly available....
there was an option to open an account....

IMO these are tragedies of Shakespearean proportions. Sure, we create "accounts" - using the checkout page and the user info. If they don't enter a password, temporary password is auto-created for them, and is sent to them with their order right under the "check the status of your order" link.

Anything you can do for the customer, you should, things you can't, you make optional, or try to.

[digitalv]

Another thing to add - go to spoon.net/browsers and use the browser sandbox. It lets you surf using all Internet Explorer versions from 6 forward, Chrome, some of the mac browsers, Firefox 1-3, etc.

Basically, all of the browsers you didn't install on your system you can just run them from here to test your site. VERY useful.

[enigma1]

Unfortunately lots of small businesses invest little time to find a framework that worth doing e-commerce with.

Turn Javascript (and Flash, if it's involved, which it rarely is) off. Re-test.

There you go, that's one of the problems, lots of so called "shopping carts" don't work (without js). And merchants do not realize to maximize their revenue they do need to support as many different browsers and configurations of browsers, as possible.

"Only works with js or flash" aren't good options for online stores, then shared SSL again if you see most carts don't work either, or they need to be on a virtual or dedicated server. Add the browsers at the top and see how many stores have problems.

I think if the merchants were aware of just the issues and tests rocknbill and digitalv mentioned, 90% of the popular carts available today would be trashed.

[digitalv]

I would agree with that on Flash, but not so much with Javascript. If a browser has javascript disabled (rare except in the case of mobile browsers) they're already going to be used to a pretty limited web experience. There is no reason to make a 1999-style website so the 5% of the people who turn off Javascript can check out, while the remaining 95% could benefit from some of the newer features out there that can set your store apart. Even the little blackberry browser supports Ajax.

As far as flash goes though, yeah ... there are a number of reasons not to go with ANY cart where Flash is an integral part of it, but not just for customer experience - aside from denying all iphone and ipad users access to your store, you can't "tell a friend" about the "part" of a Flash file you saw and liked, and the search engines will never see it.

Anyway just my 2 cents... stay away from flash carts, but don't rule out the ones that require Javascript if the features of the cart are worth it.

[rocknbil]

Many people misunderstand this statement, it's as if it means "don't use Javascript." It does not. It just means it should function as well without it, and developing a cart or checkout process that degrades gracefully is seldom difficult.

Setting aside user experience, many sites relying on Javascript will also rely on it for input validation, which often leads to hacked web sites and other nasty stuff.

[enigma1]

having a cart that requires exclusively js to run should be a no-no for merchants. You may see perhaps a 5-10% rate from various browser statistics with js off, but the reality is more people are actually controlling js on/off according to how much they trust a site. So I use noscript all the time and I enable js after I see what the site is all about. If I get a blank page, broken content or some message "you need to turn on js" I will continue to the next site to buy something. There are still quite a few security issues that I cannot consider to having js on for unknown sites.

[jwolthuis]

You may see perhaps a 5-10% rate from various browser statistics with js off

Not sure where you get the 5-10% figure, but it seems high. My site caters mainly to non-techies, and lack of javascript is never a problem.

One could make the same argument with cookies: "having a cart that requires cookies to run should be a no-no for merchants", because some people might just have their browsers set to reject 1st party cookies.

Designing a site to support Mosaic is certainly possible, but at what point are you wasting development time supporting fringe cases, instead of growing the business?

[aleksl]

rocknbil: and developing a cart or checkout process that degrades gracefully is seldom difficult.

ha-ha.

Last cart we've used we spent 4 digits to get access to source code so it can be modified. this is a top quality very advanced cart used by a few big boys. which doesn't degrade, either gracefully or not. After monkeying around with it, we were able to "hack it" so that it will actually work, but there really isn't anything "graceful" about it...considering the fact that a typical cart now has template that does most of the code for you, and you can't actually detect if user has javascript enabled via server-side code (well, in the platform we use).

I would probably have to pay $2000-5000 on top of the above to an external programmer to make the cart work without javascript if I had to hire external help. Considering that it is just a small part of a big tool...small merchants just aren't getting there.

[rocknbil]

.... developing a cart or checkout process ..... Last cart we've used we spent 4 digits to get access to source code so it can be modified.

I suppose I should clarify the context, sorry. I can't speak for "off the shelf" carts, my comments refer to when I am programming a cart for someone, these are my procedures. When I say "developing" I don't mean setting up an off the shelf cart, I mean writing the code.

One could make the same argument with cookies:

Yeah, sorry, left that off my list, and is definately part of the testing. Run it through with cookies disabled. This is a bit clunky, but it works: what you do is if you can't set a cookie, you output a warning to the user that cookies are disabled and they won't be able to assemble a cart in the normal way, they'll have to go directly to checkout or enable them with full instructions for modern browsers.

With small carts you can carry carted items throughout so it can be done without cookies, and really, it should be that way, but with larger carts and thousands of items, it's a road I haven't gone down in quite some time.

[caribguy]

I sense there could be a market for custom(ized) carts that do not depend on js/cookies. Why do the "off the shelf providers" not seem to care?

On one of my sites, I developed a custom utility to process transactions. It requires JS and (shame on me) does not degrade gracefully or otherwise. It was a conscious decision.

In more than two years, I have yet to hear a single complaint... Maybe "Joe Sixpack" is not as apprehensive about running scripts or accepting cookies as we are? Would love to see some percentage figures.

[enigma1]

One could make the same argument with cookies:

Yes he could and so is best if the cart can also operate with a session in the URL. And perhaps the server uses a shared SSL so again is best if a cart operates with a shared SSL. And so many other different configurations. And perhaps the merchant just starting up his business and he doesn't want to spend thousands in dedicated server hosting (because the cart doesn't work otherwise....)

"off the shelf" carts that's something quite vague as well as the various price figures. I could very well say I know people who spend $0 installing an open source cart ("off the shelf") through their hosts by clicking a button. But it doesn't say much. At the end of the day whatever cart you use it has to be customized and you will spend money or time/effort or both. So why ruling out potential customers because of a broken cart?

The statistics you see for the js on/off are not that accurate. The fact is more people are becoming aware how to control js and selectively can switch it on/off with the aid of various browser plugins.

IF the cart doesn't operate without js then use another cart. Because that's what belongs to the 90's. Today you have AJAX and you have frameworks that can utilize the technology without mangling the html with jscripts. Pages should display the same with or without js. And when js is on advanced features for layout, checking out etc etc can become available.

[rocknbil]

Why do the "off the shelf providers" not seem to care? (about functionality without JS, cookies, etc.)

I'm not touching that one, other than to say it's a heated discussion and many tell those of us that adhere to "accessibility for all" that we're "sooooo 1995." :-)