If you could ask your customers any question....

What questions would you want to put in a customer survey

[tolachi]

As many of you may or may not know, once you sign up with Dealtime you get a little link under your logo that let's people view your rating on Epinions. You also get the ability to add an Epinions exit survey to your order confirmation page, whose end results I assume will affect your Epinion's rating.

The Epinions survey is superior to Bizrate's, the only other I have tried, in that you can customize it with up to 5 personalized questions for both the exit survey and the follow up survey.

These questions appear to be a great way to find out what your customers want, liked, didn't like etc... But what should we be asking them?

My limited findings are that the open ended "If you have any additional comments of any kind we would enjoy hearing them." only has a 10% response rate but is pretty interesting.

"What factors most influenced your decision to purchase from us?" gets a 66% response rate and really isn't that insightful but does help confirm some things I thought were working well.

"What products would you like for us to begin carrying?" gets a 33% response rate and is potentially very useful but not so far.

I'm considering adding something along the lines of "Did you have any reservatiosn about purchasing from us? What were they?"

To me it appears that extremely open ended questions get a very low response rate. Questions that are more specific with easier answers get higher response rates.

But what should we be asking here?

Pete

[jsinger]

We did a lot of surveying of visitors to our site in the 90s and learned much. Many respondents offered long and well thought out comments. I'm surprised you don't see much surveying on commerce sites.

Now we do have a catch-all link to send comments to the company but no one uses that. Need to ask specific questions.

[ecommerce man]

I have had success with the following:

What did you like most about our site?
What did you least like about our site?

I found the answers to be very enlightening. Open ended questions can work well, just don't use too much of them.

[Perplexed]

It might be worth offering alternatives. ie, would you prefer "A" or "B" for site design and navigation issues etc.

This question put me in mind of one particular shop which the wife wants me to access sometimes. Someone has obviously told them that people do not like to scroll..... so every page on the site is only one screen deep. I would prefer scrolling down anyday to having to wait for pages to keep on loading.

[cfx211]

As someone who ran the feedback program on a small business marketplace, I can tell you the two biggest reasons people chose one vendor over the other were price and quality of the vendor.

In terms of questions to ask customers, I think that is really specific to your website and your product. If you sell large ticket items it would be interesting to know how long it took to purchase the item and what made a person choose online over offline. If it is a small ticket commodity item I would say was it only price or was anything else involved.

Finally you will learn as much if not more by surveying people who do not purchase. Don't just talk to successes, talk to failures.

[jsinger]

"What products couldn't you find on our site?"

Note that merely asking questions will impress many customers. Mention that all responses go directly to the president.

Old fashioned paper surveys cost a fortune. They can be done very cheaply on the web. You will get an enormous volume of quality respones. My wild guess is that one person in 10-100 will answer some questions depending on how prominent the survey page and how interesting it is to visitors.

You can ask for voluntary email addresses. Many will comply and that will help build your email list. (at least many complied in 1999...pre-spam days)

[tolachi]

Thanks for all the advice. Some of this I would have never thought of. The only info I have to add as of right now is that I'm getting a response rate of about 20-30%. The epinions survey pops up directly on the thankyou page. The response rate is 33% lower for answering long form questions. This is a huge improvement over bizrate which only offered a link to the surver in the popup, though there was somthing about a free something or other. My guess is that presenting the survey directly on only 1 page and keeping it relatively simple is responsible for this 3 to 4 fold+ increase in responses.

Are there any ideas on a best practice for surveying non buyers? Perhaps a popup for people with items in their cart but leaving the site?

Thanks,

Pete

[figment88]

In a former life, I was Director of Research at an online survey firm.

Originally, one of the things that the BizRate service was good at was not the POS (point-of-sale) survey but the follow-up survey when someone received their item. Shipping was and remains one of the biggest concerns for eCommerce.

As cfx211 mentioned, surveying non-purchasers is generally much more useful than purchasers. Non-purchasers, though, are often harder to get a hold of.

With your epinions POS survey, I'd focus on competition. Questions along the lines of "Did you consider making your purchase at other online or offline stores?" "If so, what made you decide on purchasing form us today?" "Have you purchased similiar items from other online or offline stores in the past?" "Have you purchased from us before?" etc.

If you can determine the factors that allowed you to convert where others failed, you can highlight them in your marketing collateral.

[cfx211]

Also if you want to be hardcore about this, I found phone interviews to be a lot better than simple email surveys. This is especially true if you are not sure what is important or what you should be asking.

Buy some movie passes, call your customers up, ask them if for a pair of movie passes you can talk to them for 15 minutes. Come up with a skeleton list of basic questions first. As people elaborate in their answer, look for followups and try to get the most out of them.

You can probably only do about 1 phone interview for every 4 online surveys. Its hard to get a hold of people and then they last a bit, but you can pick up more from them and they will help guide your online questions.

[jsinger]

If you're getting an enormous rate of people taking the survey, perhaps it's too intrusive (like a popup)and is driving some buyers away. I find many online surveys annoying when the questions are long and plain stupid as they often are. Always have a "catch-all" comment box.

A key element of surveying is that you don't need many responses to get reliable info. Those TV polling services (Nielson) use only about 1,200 participants to determine the viewing habits of the U.S. You'll have a pretty good idea of the results after a few hundred responses.

If your survey is part of your site (not epinions)there are some ways to present it to only a small % of visitors on a random basis. "You have been selected..."

-------
I didn't like to mention competitors by name in our surveys. Customers probably aren't aware of all the alternatives.

-------

Another idea (you really have me into this issue!)
State that "From August 18 thru August 25 We are doing our Annual Visitor Survey. Would You Please Answer Some Important Survey Questions")

Drives me crazy to see the same question on a site for several years. I know no one is reading the 20,000th response!

[Perplexed]

I think jsinger's suggestion for a one week blast is an excellent one. Sort of gives people the feeling of being lucky to have times their purchase so well.

[hannamyluv]

Be careful offering anything to your customers in return for answering a survey. It's been shown to skew results. People will answer your questions more positivly than otherwise because they feel either buttered up or they feel that in order to really get the "prize" they need to please you. On the other hand, if you are looking for positive reviews that will be published, it's a great way to make yourself look good.

As for questions... We do a quarterly email survey of customers who bought from us. We vary the questions each time. One time we may ask about their email preferences. Another time about site design. And maybe the next time about products. We always throw in a few demographic questions, as these help with product and design choices.

[tolachi]

Sorry to bounce this back up but, while researching exit surveys I found an article that was extremely insightful with regard to web surveys in general. It talks specifically about what increases and decreases response rates. And it talks about exit survey strategy.

[virtualsurveys.com...]

Enjoy,

Pete

[Hawkgirl]

Be careful offering anything to your customers in return for answering a survey. It's been shown to skew results.

Depends on what the incentive is and how you word the offering. In my previous life as a survey researcher, we did a lot of split-run testing to see whether the incentive influenced more than response rates. It can be quite an effective tool if you're careful with it.

[ShawnR]

My suggestion is to include questions which allow you to segment the audience, so instead of getting an aggregate of preferences, you can put it through a stats package such as SPSS (or just a spreadsheet) and figure out the preferences of different segments.

So if you have 5 qns to play with, I'd suggest 2 to segment, and 3 to determine preferences.

Sorry, this probably won't help with response rates. If the questions are too personal it will do the opposite.

[tolachi]

I'm actually going to design my own on exit javascript popup survey, cookies and all. Bizrate charges close to 20k which is out of my league. So I will have unlimited flexibility here.

Preliminarily I plan to keep it to one page with no scrollbars. I haven't thought up questions yet, but I appreciate the suggestions. I was thinking of targeting certain parts of my site initially. For example, the shopping cart, checkout process or privacy policy areas.

[Hawkgirl]

ShawnR, good point - although depending on what customer data you collect at the selling point, you may already have tons of segmentable data.

Asking demographics is great if you *need* that info. But if it won't help you sell more stuff in the long run, don't waste a survey question asking it.

For example, if you track where your traffic comes from via a unique URL, you can segment your customer base by origin.

You can segment by customer spend - are bigger spenders more or less satisfied with your site than smaller spenders?

You could segment by area of the country. If you're in the US, just chunk states into "regions" and look to see if there are differences.

The possibilities are endless. The key is figuring out what you want to know before you start asking questions! :)

[ShawnR]

"...Asking demographics is great if you *need* that info. But if it won't help you sell more stuff in the long run..."

An important thing to note is that there are many ways to segment. Demographics is probably good for FMCG, but for other sorts of products/services, you should consider whatever is appropriate to your offering.

There is always the great debate between the opposite ends of the spectrum:

• Traditional 'Marketing': Make your offering as attractive as possible to the segment/target which is most attractive to your business (e.g. highest margins, highest growth, largest volume, or whatever criteria you find attractive to your particular business)
  
• Disruptive Marketing: Make your offering attractive to a segment which doesn't exist because it is such a crappy segment (business-wise) that no one else is supplying an offering that that segment can afford to buy.
  

While there is this debate, accepted marketing wisdom and text book teaching runs in favour of traditional marketing discipline. i.e. Segmentation and then targeting and positioning does indeed help you sell more stuff in the long run.

Just to put this back into the real world: Lets say you had a question such as:

If we had to improve one aspect of our service, would it be A, B, C, or D?

Now lets say you sell your service to small businesses or individual consultants only. A 15 year old kid has no use for your service, and nor does a large corporate. Wouldn't you want to have a question such as:

Are you (A) a 15 yr old kid, or (B) an employee of a large corporate or (C) a commercial manager in a small business or (D) a consultant?

That sort of segmentation question allows you to figure out which segment prefers what, and then you can ignore the preferences of segments you are not targeting and get no revenue from.

Or did I misunderstand what you were saying, Hawkgirl?

[added: Oops, it seems I did misunderstand what you were saying. I didn't read the "if" in the second sentence I quoted. Sorry]

[aspdaddy]

>You could segment by area of the country

I think the segmenting has to be more specific to the business/product. I am segmenting orgs by postcode sectors & funding types, as I know certain ones are much wealthier - but county/city info doesnt tell us this. Another list I segment users by position in the company, and industry area.

IMO, all surveys should have a small part for demographics and open/closed questions for both stats and feedback.

[Hawkgirl]

>You could segment by area of the country
I think the segmenting has to be more specific to the business/product.

Exactly. A company I consult with sells a service, for example, that is more popular in Ohio and California. So for this business, segmenting by state and region makes sense. It wouldn't make sense for all cases.

ShawnR - we're on the same page. ;)

[Chndru]

while we are at it, does anyone know what degree to pursue (in US) if one aspires to be a market research analyst?
Is there any masters in market research or something like that? Or one has to do masters in consumer economics or m.b.a? I have a masters in engineering and fresh out of school.

Any real-life experiences are gratefully appreciated.

[Hawkgirl]

There are tons of programs out there. Check out this list [ku.edu] of survey research centers mostly set in academic settings.

The one program I've had direct experience with, the Joint Program in Survey Methodology [jpsm.umd.edu], offers both a master's program and a Ph.D. program.

Lots of places to get started in survey research - many many survey researchers have degrees in sociology. ;)

[Chndru]

Thanks Hawkgirl. I looked into those links. They tend to focus on surveys. I would ideally want to broaden my scope into market research as a whole rather than just one of the tools of doing it. I hope i make some sense to you.

Ideally i am looking for something like [bus.wisc.edu...]

Thanks :)