A/B testing is often the tool of choice for measuring the effectiveness of individual marketing campaigns.
One such example of this in practice might unfold as follows: Seeking to increase sales of a particular product, a company sends one group of customers (known as Group A) a specific offer. Then, they compare how the purchasing patterns of Group A change relative to those of a different group of customers, Group B, that did not receive the offer.
The difference in revenue between these two groups is the return on investment (ROI). This type of measurement is most effective at points when there is a clear closing date for the campaign and the results are clearly and neatly juxtaposed against a baseline.
However, for loyalty programs, actually measuring loyalty program effectiveness can be more challenging, as there is no clearly defined end in sight. Instead, they extend in perpetuity (or at least for the lifetime of the member in the program). In these types of scenarios, companies find themselves needing to influence behaviors over an indefinite period of time, rather than aiming to simply cause one particular response at a given moment.
The crux of the problem with measuring loyalty program effectiveness is that it’s hard to predict consumer behavior over the long term. In order for companies to adequately gauge ROI, they need to correctly forecast how the loyalty program will impact the frequency of desired customer behaviors in the future, and not just rely on historical data.
So, what exactly should loyalty teams be measuring when determining loyalty program ROI? Let’s take a look.
A commonly-used method for understanding incremental lift, the “look-alike analysis” is vulnerable to self-selection bias and informational blind spots. However, there are other approaches that produce superior results.
In order for companies to rate the efficacy of a loyalty program, they have to be able to chart out its effect on consumer behavior. As mentioned earlier, it's important to note that I'm referring to future behavior, not past performance. While prior behavior can provide valuable historical context, the scope of its use is rather limited.
One example of the limitations seen in these “look-alike” analyses (of which I've observed many companies employ to quantify the incremental lift created by customer loyalty programs) is the blind spots in behavioral profiles which result. Incremental lift, of course, refers to the increased amount of spending in which a member engages above the quantity that they would have leveled-off at in the absence of the loyalty program.
A look-alike analysis works in the following way:
Customers are grouped based on the characteristics they display when they first join the program. One such group, for example, might comprise 40 middle-aged men that joined the program with the company through Channel Z.
This group can then be divided into two distinct subsets: those that choose to sign up for the loyalty program, and those that do not. The company then follows these customers throughout the course of a predetermined period of time and compares the volume of spending between the two sub-groups. The estimate of the incremental lift generated by the loyalty program can be found by looking at how much more the group of people enrolled spent compared to those that were not enrolled.
This approach is beset by two significant problems, however. The first problem is the challenge inherent in tracking customers who are not enrolled in the program. For most programs, customers who aren't enrolled simply cannot have their behavior monitored. The second issue is the self-selection bias that accompanies this method — that is, customers that choose to join are inherently more likely to continue spending with a company (otherwise, why would they join?).
I've seen this self-selection bias express itself powerfully, resulting in unrealistic and unviable estimations of incremental lift. In fact, I've even experienced some companies abandoning this strategy outright because the answers it provided seemed so unreliable.
There are, however, better ways of assessing the utility of loyalty programs. In particular, predictive “future value” metrics are very useful. These are metrics that predict the future value of each member. My three favorite predictive future value metrics are as follows:
Customer lifetime value (CLV)
Customer lifetime value (CLV) calculates the figure for how much free cash flow any individual member creates over the course of their lifetime within the program. For many loyalty programs, CLV could be determined using this formula:
CLV is most valuable when you can estimate it at the individual member level. You can then aggregate this information based on the channel in which the member joined to easily discover the value generated from each channel. Assuming the amount is both large and positive, then you can assume that a particular channel of acquisition is effective and reliable (and breathe a sigh of relief).
Moreover, a CLV that gets larger as time progresses is a fantastic indicator of performance when evaluating whether or not a loyalty program is succeeding at bringing about an increase in customer engagement.
Customer future value (CFV)
With this metric, we measure the expected future free cash flow produced by each member. As you may have guessed, it focuses on future member behavior.
Though, superficially, this metric resembles CLV, it restricts its focus to upcoming free cash flow. Previously-generated free cash flow is not integrated into the analysis. This approach is great for evaluating enrollees during their lifetime with a loyalty program (in contrast to at the moment of acquisition) due to the fact that it concerns itself with a future value that can be influenced. For healthy programs, this metric should be large and show an upwards momentum (though, it will probably bump into an upper limit, depending on your business model’s parameters).
CFV should be estimated for each individual member. This enables the identification of members with the most significant future value, and turns the focus of targeted marketing on them. In this way, your company will reduce the amount of investment wasted on courting members that don't produce returns, ultimately driving an increase in long term financial value
Customer potential value (CPV)
Though CFV is useful for targeting and identifying which members hold the most future value, it's also important to identify which enrollees have the potential to convert into high CFV customers if properly incentivized. These individuals are called “high potential members,” a name that distinguishes them from their “high value” counterparts.
You can measure this by zeroing-in on the marginal change in future value for additional earned points. Members with high potential are those whose CFV goes up with every new point they accrue. Those with a positive CPV generate positive value every time they earn a new point, whereas those with negative CPV values cost more than what they bring in every time they tally up another point.
CPV is helpful for optimizing a loyalty program’s financial value. If we recall from entry-level economics, profit reaches its peak when marginal cost and marginal benefit are equal (or when the value of marginal cost subtracted from marginal benefit equals zero). This signifies that optimal levels of long-term profit are attained by a loyalty program that has a CPV of zero. Though a CPV of zero may not be practical, an understanding of CPV is helpful for advancing your company closer to a theoretically optimal position.
Once again, if your company has an estimate of individual-member-level CPV, it will be able to identify members with the highest potential and aim its targeted marketing towards them. Similarly to properly estimating CFV, doing so will bring down the marketing waste incurred by engaging members that don't create returns, and drive long-term financial value.
Bottom line: While a bifurcated, “look-alike” analysis leaves companies with blind spots in both their predictive scope and ability to capture ongoing customer behavior, there are other approaches that give them the information they need to evaluate and optimize the value of their loyalty programs.
Customer lifetime value (CLV) informs a company of how much a member will generate over the course of their lifetime in the program, including value already created. Customer future value (CFV) addresses the future value a member will bring in, which excludes value already created. Customer potential value (CPV), in turn, tells companies which members are likely to convert into high-value contributors. Knowing these figures allows companies to reduce how much money is wasted on engagements with little-to-no returns.
It's important to focus your company's investigative efforts on predicting value at the individual member level in order to not miss out on opportunities to increase revenue.
It's shocking how frequently these predictive future value metrics are not developed or taken advantage of.
It seems that a large percentage of companies fall short in their efforts to correctly prioritize CLV because of the obstacles inherent to predicting these numbers.
To correctly predict CLV, a company must able to predict behaviors (e.g., buying patterns by type, volume and frequency of purchases, how often a member redeems) at the individual level, and, more importantly, correctly anticipate how they'll evolve far into the future. Long term prediction is, of course, very difficult.
However, once you're able to surmount these formidable obstacles, you'll find your company in a very favorable financial standing. Not only will your company have a financial model that clearly details how much your company can invest per member without sacrificing economic viability, you'll also be able to group together members in a precise, strategic fashion that increases the effectiveness of your targeting efforts.
A number of loyalty programs use CLV metrics, but they tend to suffer in one key dimension: redemption cost.
Frequently, these CLV models incorporate redemption cost assumptions based on the breakage models used for financial reporting of the liability. Typically, this breakage assumption is a single aggregate number for the entire program. The CLV models apply this aggregate breakage expectations uniformly across all members.
Of course, this is not a realistic assumption, since there are marked differences in breakage between members. In fact, a commonly-touted statistic is that only a meager 20% of members are responsible for 80% of redemptions. This indicates that employing an assumption hinged on aggregate breakage figures will not accurately reflect the behavior of most members.
Instead, such an assumption will depress CLV and overstate costs for the majority of your members. As a result, a significant portion of your members will have their estimated future value diluted, distorting the economic picture and leading to missed opportunities.
Consequently, the first step towards developing these metrics is to thoroughly understand loyalty program liability. As I stated earlier, the most important driver of the cost component of CLV, CFV and CPV for loyalty programs is the cost of redemptions. Companies need to know how to properly estimate ultimate redemption rate (URR), liability, and cost-per-point (CPP) at the individual member level to get started.
Bottom line: In order to get the most out of these predictive future value metrics, you have to use them correctly — and precisely. Using aggregate rates of breakage will almost certainly misrepresent the value of each customer and produce inaccurate conclusions. This leads to companies missing out on opportunities for unlocking potential revenue by engaging members who actually are, or could be, converted to high future value members.
Instead, companies must have a well-defined understanding of their breakage rates at the individual member level in order to produce predictive valuation levels that steer your business towards optimized member engagement.
If you want to know which customers to engage, having accurate predictive “future value” metrics to evaluate the lifetime, future and potential values of individual members is essential. It's important to derive these metrics at the individual member level so members can be differentiated and prioritized.
When measuring loyalty program effectiveness, the biggest metric companies fail to properly estimate is the cost of redemptions. Companies that use aggregate cost assumptions in their predictive value models can find themselves overestimating redemption, therefore drawing distorted financial assumptions for most of their members. This results in missed opportunities to drive value. If you were to multiply this by the entire membership of a loyalty program, these missed opportunities can culminate in millions of dollars worth of atrophied growth.
Correctly mining data provided by customer behavior is the most salient method for companies to get the individualized predictions of behavior and capitalize on the chance for increased value. Make sure to read my next piece to discover valuable insights on how to best project the future actions of customers at the individual member level.
Comments