“Moneyball” for Business – Why Your Company Needs Customer Experience Metricians™
The current state of customer experience is largely focused only on the online components of a customer’s interaction with a company. While this is totally understandable given where the term came from and why (see post, “Where Customer Experience Came From”), the fragmentation taking place for consumer’s time, energy and money is impacting all aspects of every business, not just online.
When you think of an old, traditional business format what comes to mind? If you guessed grocery stores, you have a perfect example of what I mean.
Long ago, a program for predicting new grocery store location sales growth was developed. It was called a “gravity-weight store sales forecasting” model. The theory for its predictive ability was two-fold:
1) The underlying premise was that every geographic area’s consumer demand for grocery goods was driven by their distance to a particular store, accounting for ingress and egress issues that may occur in a specific traffic pattern to any one store. If you think about every geographic area as a bucket full of water and all the grocery stores in the area as sponges, then you get the idea that the model was predicting how many sales (water) each new store to an area (sponge) could soak up.
2) This model proved to be very predictive in those areas of the country where the geography was flat or relatively flat and all you needed was a census map to determine population sizes within 1, 2 or 5 mile radii of a new store location. The second reason for the predictive ability of this model was its ability to be almost continuously updated when new information occurred such as competitive reaction. Therefore, the judgment of its success as a model was a pretty even split between the mathematics of the modeling process itself and the ability of the technician operating it to accurately account and accommodate changes in circumstances.
Long story to get to the earlier point; consumer fragmentation. Driven almost entirely by changes in attitudes and behavior caused by the internet’s influence, consumer fragmentation placed the usefulness of even this grocery store modeling and forecasting tool in jeopardy. Even in flat geographic areas, consumers are no longer acting in a predictable manner. They are changing patterns held for decades with respect to their grocery store buying.
Why the need for a new type of strategic, mathematically proficient, business-minded and tactical person in any business seeking survival, let alone success, in the coming decades? Because the customer has changed. It’s permanent change continuously fostered through technological advancement. To most people its unpredictable change. One thing is for sure, it is change that will happen more in the future, not less.
“Moneyball” for business
One of the things I believe firmly is that truth can be derived and shown through multiple sources and methods. It is not relegated to one particular area of life or dependent upon any particular person’s interpretation. Truth is universal.
If you have not already read “Moneyball: The Art of Winning an Unfair Game” by Lewis, you should. Not only does it provide more insight into the thought process behind a professional sports team (the Oakland A’s) than ever previously published, it also provides a glimpse into what will be the cornerstone for successful businesses moving forward: hardcore statistical analysis on metrics determined by analysis to be meaningful to a corporation’s goals.
This method of making decisions in the professional sports business has permeated more than just the A’s in baseball. This year’s Houston Rockets are an example of how use of these processes completely yield “expert opinion” meaningless. Of all NBA professional sports franchises, the Rockets are the heaviest and most committed applicants of the statistical analytical theories put forth in Moneyball.
When Yao Ming, the Rockets All-Star center, went down with a season ending injury, not one professional sports expert or pundit gave the Rockets a chance of making this year’s playoffs. Not ONE!! Let that sink in for a moment because it’s very enlightening if you think of what business experts or pundits proclaim everyday regarding companies and their future chances of success.
Not only are the Rockets secure in making the playoffs, they continued the winning streak they began with Ming in the line up and rattled off 22 straight wins, the second longest streak in NBA history. In sports circles, credit is being given (albeit very grudgingly) to the organization’s application of the same principles of statistical analysis demonstrated in Moneyball. What else can they give the credit too when outcomes defy all conventional wisdom?
This is a perfect example of what customer experience success™ is for business. And, as in Moneyball, a new process requires new expertise.
The three levels of Customer Experience Metrician™
There are three personnel needs facing companies today and the vast majority of them do not know it yet. These are:
1) The Enterprise Customer Experience Metrician™
2) The Strategic Customer Experience Metrician™
3) The Tactical Customer Experience Metrician™
Let’s begin at the highest level, that of the Enterprise Customer Experience Metrician™. The ideal qualifications for this position are: high level of advanced mathematical aptitude, ability, and application of mathematical processes and models; experience starting and running an entrepreneurial business (not a franchise) as well as a department or function area within a large company (ideally Board level expertise); business financial aptitude and understanding of financial analysis applications; ability to correlate a business’ goals or aspirations to innovation, intellectual capital, communications, resources, culture, leadership, execution and strategic alliance opportunities; and finally, the ability to fully understand the application options and oversee the implementation of all potential components for a full customer experience process, integrated seamlessly within the fabric of a company in order to produce the necessary on-going knowledge for a company to make business decisions.
For the Strategic Customer Experience Metrician™ the focus tightens to that of the motives of the company, those motives that drive options which determine the setting of measurable objectives. This position again requires a high level of advanced mathematical aptitude, similar to that of an actuarial in nature. This then must be combined with incredible people skills of leadership, innovation, communication, team building, and the ability to guide others into strategic thinking regarding all elements of their business, across company departments and silos. However, strategic leadership is the hallmark of this person’s ability. Fostering a consistent understanding of the optimum strategy developed and bought off on by the necessary management and leadership teams. Translating this strategic direction into defined plans, and overseeing its implementation all the while anticipating and identifying where change is needed to address future issues.
Last but definitely not least is the role of the Tactical Customer Experience Metrician™. Many companies have mini or parsed versions of this role within their companies or they rely on small bits of what is necessary within this role to be supplied from third party industry sources. The number of companies I know that actually have the internal manpower for this role is incredibly small.
The reason is this role covers the day-to-day application of all methodologies, processes, data mining warehouses, and dash-boarding necessary to deliver metrics needed for successful business decision to take place at the highest level. This can be an overwhelming task for all but the largest companies to handle in-house, hence the need for outside resources skilled in all aspects of this role. Most third party industry sources provide only small glimpses of the information necessary for a company.
An example of how fragmented the market has become for those attempting to keep companies up to date on tactical metrics is easily seen when considering vendors supposedly supplying hard metrics to companies for understanding the effectiveness of their website. It is not unusual for a medium size company to have seven or more such external vendors, each supplying either a different look at their website’s hard metrics or a unique report on one specific website functionality. No wonder it is referred to as a breadcrumb meatloaf of metrics. The key problem underlying any company’s ability to take serious, let alone act on much of what is produced by these vendors, is the fluidity of interpretive change within the internet industry. While page views were “the metric” last month, this month it is invariably something else, equally unproven to actually correlate to any real business goal or objective.
Back to the Tactical Customer Experience Metrician™. An incredibly strong mathematics and statistics background is a must for this role as is an understanding of programming languages, statistical software and all basic workplace applications software. This role either directly manages or performs data analysis, development of algorithms, models, and computational designs. Obviously, this role must thoroughly understand all elements and components of market research, usability formats and testing alternatives, analysis of small to large-scale datasets, application of advanced analytics, use of experimental research design, and interpreting hard metric output (such as that derived from website analytics). They must implement, document and present technical information in human terminology to a broad corporate audience. Ideally, they will have industry benchmarks from which to refer the client to for understanding their actual market position relative to individual findings. Finally, this role applies tools, either third-party sourced or proprietarily developed, to produce real client data, in the end presenting the results of the modeling solutions to all necessary areas of the business.
Some, but not all, of the customer experience feedback areas covered by this role include: brand (among all channels the company uses, within a comparative competitive environment); attitudes, behavior and usage analysis (current and projected); market segmentation (on-going, incorporation of personality level – we call ethos/values – of consumer choice modeling); channel benchmarking and tracking using statistically meaningful, reproducible and actionable methods (includes online – current verbiage closest to what we are portending is web analytics – retail, catalog, etc); advertising and marketing campaign benchmarking and effectiveness tracking measures; and finally, business activity tracking – those key actions taken by customers that impact the business, from the simple area of sales analysis to the more complex.
The Customer Experience Metrician™ and Business Objectives
Based on years of research and consulting, we have found that while every business claims to be totally unique or face problems only applicable to themselves that is simply not true. Regardless of the industry, we have identified six umbrella business objectives that companies focus on at all times despite varying their immediate individual importance to business operations. These are:
1) Revenues
2) Profits/ROI
3) Customers
4) Employees
5) Products and/or services
6) Stakeholders
While the order of importance may change and some companies may place different monikers on these six objectives, we have yet to find objectives that do not fit within one of these six. Understanding the impact of each of these six objectives throughout all areas of a company is critical to a Customer Experience Metrician’s potentially successful implementation of a customer experience success™ program.
Each of these six objectives carries an impact throughout all areas of a company. Areas such as; sales, pricing, operations, customer service and support, advertising, marketing, human resources, etc, are affected differently depending on which objective lens they are being viewed through. The Customer Experience Metrician™ must understand this.
The Customer Experience Metrician™ and Metrics
There are two types of metrics that the Customer Experience Metrician has to be completely knowledgeable of:
1) hard metrics – defined as that metric from a server or software (think page views as an example)
2) soft metrics – a customer or potential customer’s direct response to a specific query (think survey or usability)
Hard metric suppliers for the online world are proliferating at an amazing pace. I will attempt to devote an entire future post just to this area and what I know personally of some of the companies within it. It is sometimes amusing to see the claims which providers of these types of metrics make to the corporate world. Some are reminiscent of weight-loss product claims as to the amazing impact the implementation of the product will have on a company’s life.
In truth, hard metrics have always been around. In retail for example, there are myriads of hard data from sales to warehouse movement reports to customer traffic pattern analysis captured either electronically or observationally. Then, as now, hard metrics are incomplete and need the value of soft metrics in order to direct a company’s actions. We like to call it the footsteps in the forest problem:
I like to use the following visual definition of customer experience soft metrics to help someone understand the dynamics facing just a company’s website.
When you consider what is required here to produce quality, actionable information for good business decision-making regarding a company’s website, you quickly see that none of these critically important questions are answered by hard metrics. Nor can they ever be!
Soft metrics include both qualitative and quantitative methodologies, each having their own place depending on the company objective, the area under study and around fifteen other triggers for deciding which approach will be used.
Qualitative methodologies include focus groups, usability testing (both in-lab and using remote usability techniques such as we developed at NetRaker), triad groups, and follow-me techniques are some examples. These techniques in the hands of a skilled Customer Experience Metrician™ are as good at producing insight today as they were when they were first used in consumer product research many years ago.
Quantitative methodologies include larger, projectable sample sizes (at least 50 or more respondents required per cell within each information matrix) and include: telephone interviewing, in-person interviews, large-scale mystery visitors or shoppers (either in-person or electronically) and of course online. Online can be used for both research oriented needs as well as large-scale diagnostic applications (using tools like remote usability only applied to other objectives).
A Customer Experience Metrician™ has to be skilled in all facets of these applications.
The Customer Experience Metrician™ and Variables
There are two main types of variables which impact a company. These are:
1) Exogenous and,
2) Endogenous
Exogenous variables are those that take place outside of direct company control. The current market situation is such an example. There is very little companies can do themselves to control the fact the country has slipped into a recession. Companies have to respond to exogenous variables.
Endogenous variables on the other hand are those the company can control. These include such meaningful things as the amount and location of advertising or marketing campaign expenditures. A company can proactively direct endogenous variables with the obvious objective of positively affecting company goals.
While these examples are relatively obvious to anyone reading this in terms of the level of control a company can exert upon either of these variables, many variables companies find themselves facing or being introduced to are not so obvious. Again, this is one more area where the Customer Experience Metrician™ must have the knowledge and understanding to guide a company’s efforts toward variables they can best control with the objective of achieving optimum effect on those variables.
Is there reason enough for a new role and new skill set requirements? Just like Moneyball for sports businesses required a break from the established manner of thinking, so customer experience success™ requires a break from our current fragmented manner of addressing company’s information needs for modern day decision-making. Just like Moneyball required a different set of skills and application knowledge, so customer experience success™ requires a deeper level of expertise in many more facets than has been required to date.
Perfecting the Customer Experience 
I was on Yahoo and found your blog. Read a few of your other posts. Good work. I am looking forward to reading more from you in the future.
Tom Stanley
Tom Stanley
March 27, 2008