More Info

THE METRIC

QUERESTION STRUCTURE

LIBRARY OF CATEGORIES AND QUESTIONS

OVERALL SATISFACTION

PERFORMANCE CATEGORIES

THE METRIC
Think back to surveys you have seen over the years and consider the metric, or response options that were employed. Remember seeing surveys with an alpha scale (Please rank the following from A to F, with A being the best .......) or a numeric scale (Please rank from 1-10, with 10 being.....)? Both are commonly used but flawed approaches in which respondents tend to be subliminally driven, their responses influenced by their experience with school grading systems. Where an alpha scale is employed, a “C” is usually seen as“average”, just like it was in school. On a numeric scale, the yardstick for “average” will usually be based on a percentage; 80% being considered middle of the road, and anything below 60% considered a failing grade.

Yet, companies will look at a survey score derived from a 10-point numeric scale and conclude that an average of 5 (or 50%) is “middle of the pack”. In reality, they are probably looking at a measurement that indicates abject failure. Other scales may use various labeled derivatives - Good, Fair, Bad, Poor for example, or Meets Expectations, Does Not Meet Expectations, and so on. The problem there, of course, is that none of those carry a precise definition, so how can they be expected to produce a precise set of responses or lead anyone to a precise set of conclusions?

The metric used in any survey must be clear, concise, unambiguous and non-subjective. Regardless of region or language, you must be confident that all customers, regardless of location or language, are seeing, understanding and responding to the exact same thing.

The InfoQuest metric, shown below, takes care of that.

The anchor point, Totally Satisfied, is a “clear line in the sand” metric. There is no subjectivity, no
interpretation, no ambiguity. It’s been proven to be multi-cultural, having been used successfully in 77 countries and 25 languages. It does not stack the deck in one direction or the other, nor does it provide a“neutral” response along the lines of “no answer” or “don’t know”. Those kinds of options give people a cop-out response, an easy way to duck difficult or uncomfortable questions. Insufficient Information to Evaluate, on the other hand, recognizes a lack of experience response without providing an open bail-out option.

QUESTION STRUCTURE
Next up is the question set. There are two critical elements that need to be provided here. 

First, you need benchmark performance scores. Survey questions must work to clearly tell you where you are so that you can assess precisely where you need to go.

Second, they need to provide direction on how to get there.

Our approach to meeting those objectives is to use the structure shown below. Pictured is what we
refer to as a “Category”, which as you can see is comprised of a satisfaction question that will give you a benchmark score in (in this example) Sales Performance. Then, following the satisfaction question are three statements which collectively serve to operate as drill-downs, helping you to understand why customers are satisfied or dissatisfied with the accessibility of your sales reps.

The average survey gathers feedback on anywhere from 8 to 15 such categories, any or all of which may entail a greater or lesser number of drill-downs.

THE LIBRARY OF CATEGORIES AND QUESTIONS

Originally introduced in 1995 and updated several times since, the InfoQuest Question Library contains 63

different proven and tested Categories touching on all of the business to business basics, including such things as Sales, Product Quality, Delivery, Customer Service, Tech Support, Billing & Invoicing, Credits & Returns, Management Interactions, and so on.

The Library is made available to all  clients in an electronic format that enables distribution and review throughout   our company. Want to modify or customize some or all of the Categories? No problem.

Want to design new Categories, perhaps touching on a proprietary product or service?  Again, no problem. Or charge.

The development and use of the Library was initially motivated by the desire to reduce costs. In the early days we spent time and money with company after company designing the content of surveys that inevitably ended up bearing a striking resemblance to each other.

There are market research companies out there that continue to take the same resource draining approach. Today, that time and money can be put to far better uses. While every survey is fully customized, the vast majority of our clients find everything they need right here.

 And yet, some of the same folks who think it’s a good idea to front-end load the metric also tend to think it’s a good idea to front end load the survey itself. How?  By putting the overall satisfaction question at the front of the survey. The professed logic is that doing so will stimulate a more top-of-mind, instinctive response.

Flying in the face of that thinking, test studies have shown that placing the Overall Satisfaction question at the beginning of the survey will produce a score 20-25 points higher than placing it at the end. In other words, simple placement has a substantial impact on the outcome.

Does it matter? Well, consider this. .

Studies going back almost 20 years have consistently shown that over time, Totally Satisfied customers spend over two and a half times what Somewhat Satisfied customers spend, and roughly fourteen times what Somewhat Dissatisfied customers spend.

So yes, it matters. It matters a lot. The bottom line is, while placing the question at the top of the survey will produce a result that will look better to the board or the stockholders, placing it at the end will give you a much more reality based outcome.

OVERALL SATISFACTION

Nearly every survey, regardless of methodology, will ask a question designed to measure respondent’s Overall Satisfaction with the company or product being assessed. In our application, the specific question used is – On an overall basis, how satisfied are you with our company?

It’s a pretty straightforward question, and building it into a survey would seem to be a pretty straightforward objective. But amazingly, both companies and market researchers often manage to get that simple task wrong, sometimes in more ways than one.

Big Mistake Number 1 - Putting the question in the wrong place. 

It is most advantageous to place the overall satisfaction question at the end of the survey, thus assuring that before respondents consider the “big” question, they will have been exposed to all the other questions on all the other topics that comprise the business or product relationship. In other words, to make sure that the response provided is an informed and well-considered one.

Big Mistake Number 2 - Composite Scores

There are companies that like to take all of their “positive” responses, add them together and proclaim that (using the example to the right) “92% of our customers are satisfied”. While that provides a nice foundation for a bit of public relations bluster, let us say again what we have already said.

 Over time, Totally Satisfied customers spend over two and a half times what Somewhat Satisfied customers spend, and roughly fourteen times what Somewhat Dissatisfied customers spend.

What that means, once again, is that adding the scores together is utterly delusional. Sure, it will produce a result that will look better to the board or the stockholders, but it will very clearly NOT give you a view that is based on anything which resembles reality.

Big Mistake Number 3 – Omission

The only thing worse than misusing the overall satisfaction question is to not use it at all. There are firms out there that try to hang their hats on other “critical” factors as part of their efforts to sell you a survey.

They may use a loyalty question as their statistical anchor, or the sum of categorical scores, even some kind of  “proprietary” (albeit arcane) mathematical formula. Just remember that there is nothing to support such tactics as sound, appropriate or - worst of all - accurate. Totally Satisfied customers spend more money with you than your other customers. To grow your revenues, you need to know who falls into the ranks of Totally Satisfied, who does not, and what needs to be done to move everyone else up.

PERFORMANCE CATEGORIES

Overall Satisfaction is the single most important measure of customer satisfaction.  Totally Satisfied customers tend to be fiercely loyal and virtually immune to competitive influence, while any customer who is less than Totally Satisfied is, to some degree, at risk of loss.

Performance categories vary greatly from one company to another. While things like quality, delivery and customer service are constants for nearly any business, the type of business will determine what else needs to be measured. Often the data will also need to be segmented, to enable views by region, customer type, product line, and/or over time, to name a few options.

With rare exceptions, between 8 and 15 categories - or 32 to 60 questions - is adequate to assess the performance of a company. Fewer questions will provide inadequate data to work with. One can always think of more things to add to a survey, but it’s vital to remember that if you cross the line from survey to endurance test, the response rate will begin to fall. But, if you control your appetite, maintain the proper range of questions, and be certain you have the ability to segment the results, you will have powerful tools with which to strengthen and grow your business.