Catalog And Retailer Differences In Matchback Strategy And Contact Strategy Optimization

There's this huge shift in multichannel marketing strategy in recent years, with catalog matchback algorithms playing a significant role in the shift.

Fashion retailers (Neiman Marcus, Saks, Bloomingdales, Nordstrom) either eliminated traditional catalog marketing programs, or are in the process of significantly reducing circulation. Folks at Williams Sonoma are significantly trimming circulation.

When I talk to some of you, you tell me that these folks can cut circulation because they are retailers --- the retail channel somehow generates brand awareness that fuels a brand in a way that minimizes the need for advertising. You might be right, we simply cannot test your hypothesis.

Mechanically, retail brands are better at developing a testing discipline.

Here's an example. We randomly sample twenty customers, ten receive a catalog, ten do not, and measure performance across channels during the three weeks that a catalog is active. Here's what we observe:

Mailed			Holdout
Cust 1	Buy Store		Cust 11
Cust 2			Cust 12
Cust 3			Cust 13	Buy Online
Cust 4			Cust 14
Cust 5	Buy Phone		Cust 15
Cust 6	Buy Online		Cust 16
Cust 7			Cust 17
Cust 8			Cust 18
Cust 9	Buy Online		Cust 19
Cust 10			Cust 20	Buy Online

Here's the fundamental difference between the retailer and the catalog brand.

The retailer will compare the mailed group and the holdout group. In the mailed group, four out of ten customers responded --- in the holdout group, two out of ten customer responded. The retailer calculates response as (4 - 2) / 10 = 20%.

The cataloger does not execute the test. Instead, the cataloger takes the mailed group, identifies the four responses, matches the responses back to the mail file, and calculates response as 4 / 10 = 40%.

Again, notice the significant difference in response, using the two methodologies.

• Retailer = 20% Response Rate.
• Cataloger = 40% Response Rate.

In this comparison, the organic percentage is 20% / 40% = 50%. Half of the demand would happen without any advertising.

This fundamental difference in approach causes a shift in strategy.

• Retailer = Cut Circulation, Re-Allcoate Marketing Dollars Elsewhere, Learn!!
  
• Cataloger = Maintain Circulation, Ask For Additional Funding For Online Marketing, And Significantly Over-Spend In The Catalog Marketing Channel, Driving Down Profit.
  

This problem is systemic across the catalog industry. Matchback vendors aren't trying to rip you off, they simply aren't. But there isn't an incentice to create a "best practice" that accounts for the differences that retailers observe when executing contact strategy testing and what catalogers measure via matchback analytics.

A simple solution for catalogers is to execute a test similar to the one designed above. Do not tell the matchback vendor about the holdout group. Have the matchback vendor run the control group through the matchback algorithm, and see how many orders are allocated to the holdout group. Subtract the results of the holdout group from the results of the mailed group, and you have true incremental demand as illustrated in the retail example at the beginning of this post.


Hillstrom's Contact Strategy Optimization: A New E-Book.

The Most Important Catalog Marketing Metric: Organic Percentage

The most important metric in catalog marketing is called the "organic percentage".

The metric is defined as the percentage of demand, at a segment level, that would occur if no catalog mailings were delivered to a customer.

Most of the catalogers I speak with assume that the organic percentage is zero --- in other words, if catalogs were not mailed to a customer segment, the segment would not spend any money.

Of course, this assumption is false, perpetrated by biased matchback algorithms that incorrectly assign online orders to catalogs mailed to the customer, when in reality, the catalog had nothing to do with the generation of the order in question. You'll know that your matchback results are biased if the percentage of demand you add on to your acquisition segments (after matchback) is significantly lower than it is for housefile customers.

Catalogers who attempt contact strategy tests, say over a three month period of time, find relationships like this.

• Telephone - Only customers have an organic percentage around 10%.
• Telephone + Online customers have an organic percentage around 25%.
• Online - Only customers have an organic percentage around 40%.

In other words, if no catalogs are mailed to an online-only customer, the online customer will still spend 40% of the demand they would spend if they are mailed all of the catalogs during the quarter.

The organic percentage metric is critical, because it dramatically impacts your calculation of profit and loss. If you have a high organic percentage, then you are significantly overmailing customers, regardless of what your matchback analytics vendor tells you. If you have a low organic percentage, then you have no choice but to mail catalogs in order to generate demand.

The image at the beginning of this post shows the difference in profitability for the same segment of customers, comparing a 10% organic percentage to a 40% organic percentage. The ten percent level requires four mailings per quarter. The forty percent level maximizes profit at just one mailing per quarter. Think about what you could do with the expense from the three additional mailings?

If there were just one metric I'd ask catalogers to track at a segment level, during 2009, it would be the "organic percentage" metric. Knowing this metric fundamentally changes how you decide to contact different customer segments.

How important is this percentage? Take a brand like Nordstrom. This is an $8.5 billion dollar business that is luck to generate ten percent of that total from marketing activities. Therefore, the organic percentage is ninety percent. This brand generates ninety percent of sales without the aid of traditional marketing activities. That's a strong brand.

Think about Zappos. There's the volume they generate due to online marketing and search marketing, and then there's the volume they generate via word of mouth. I'd guess that half of their volume happens without the aid of marketing, plus or minus twenty percent.

And then think about a traditional cataloger. The traditional cataloger believes that the vast majority of demand happens becaue of catalog mailings. If mail/holdout tests validate this, then the cataloger is at the mercy of catalog marketing --- if customers are no longer responsive to this form of marketing, demand dries up.

The goal, of course, is to build a brand that has a high organic percentage, not needing advertising to drive sales and profit.

We can learn how much of customer demand is generated by advertising by executing thorough mail / holdout tests, in both catalog marketing and e-mail marketing.

Matchback Bias

You're probably partnering with your list organization, data warehouse vendor, or co-op on the never-ending scope of matchback analytics.

The goal, of course, is to prove that catalog marketing is a vital piece of the modern marketing puzzle. You're trying to truly understand the ROI of this activity. That's why you try so hard to attribute every online order back to one of the dozens of catalogs you mailed in the past year.

Now let me ask you this.

Do you go through the same effort to attribute every phone order back to the original online source?

You don't?

I met with a business that is doing just that. They combine their web analytics tool and their matchback analytics platform to attribute phone orders back to the online marketing activity (which is usually organic/natural search) responsible for driving the phone order.

Why is it that our industry is so bent on proving that catalog marketing drives online orders, but doesn't invest the energy to prove that online marketing drives phone (and store) orders?

Our view of the world is biased, folks. And that bias favors co-ops, printers, the USPS, the paper industry, and the list rental/exchange industry.

Your thoughts?

Who Benefits From Flawed Matchback Analyses?

This is my final discussion about matchback analyses for awhile, as I'm sure many of you are ready to read about other topics. But I got chewed on, I was told to stop talking and get on the multichannel bandwagon. That bandwagon may be financially hurting some catalogers. Somebody needs to talk about that.

Let's think about the industries that benefit from incorrectly executed matchback analyses. Mind you, I'm not picking on any specific individual --- I've observed many folks in these industries who counsel clients in a positive way.


The USPS: Ever wonder why it seems like there are more catalogs in your mailbox these days, compared with a decade ago? Here's a secret ... if you mail every one of your internet buyers a catalog, a matchback analysis might tell you that the catalogs drove all online orders within twelve weeks of the catalog mailing ... even if search and e-mail marketing and organic demand were truly responsible for the orders. The USPS (and now the good folks in the UK as well) commission studies that "prove" that catalog mailings drive online orders. I'm not saying catalog mailings don't drive online sales --- I'm just saying we are significantly overstating the importance of catalog mailings via flawed matchback algorithms.

The Co-Ops: Catalogers love co-ops these days. Catalogers get names that perform better, and get them at a lower cost than via list vendors. So co-ops have a financial incentive to promote flawed matchback algorithms (though some truly try their hardest to do a good job). By "proving" that catalogs drive online orders, catalog clients order more names from the co-op, driving co-op sales and profit. An even bigger conflict of interest occurs when co-ops actually execute the matchback algorithm for the client.

Merge-Purge Houses: The cataloger gets matchback results from the co-op, orders more names, names that are merged at the merge-purge house, driving increased sales and profit for the merge-purge house. Also, many merge-purge houses run matchback analyses for catalogers, earning $$$ for their efforts.

Printers: If catalogs are "proven" to drive 70% to 80% of online sales (which does occasionally happen, but not as often as we're being told), then printers benefit, too. The cataloger mails more catalogs than they normally would, which drives sales and profit for the printer. If the printer delivers catalogs deep into the mail system, then the printer can earn more $$$ too.

Paper Industry: Some of my feistier conversations have been with folks in the paper industry. More catalogs means more paper, which means more $$$ for those in the paper industry.

List Industry: I'm much less critical of the list industry, because by and large, these folks acted with integrity for the past decade, recommending that clients shift names from lists to the co-op industry, knowing all-too-well that it would result in the death of the list industry. But flawed matchback analyses help those in the list industry as much as they help the co-ops.

Trade Journals: We read about multichannel marketing and matchback analyses in trade journals. These publications depend upon the vendor community for advertising revenue. The vendor community depends upon the trade journal to "get the word out". This symbiotic relationship benefits from promotion of matchback analyses that may not accurately reflect the "truth".


So, let's look at the ecosystem that depends upon matchback analyses that are sometimes flawed.

Co-ops and merge-purge vendors do the matchback analysis, attributing too many online orders to the catalog channel. This causes the cataloger to order more names from co-ops and list vendors than they should, financially helping co-ops and list vendors. These names go into the merge-purge process, financially helping merge-purge vendors. Next, the names go to the printer. Paper reps financially benefit from over-mailing, as do printers. The printer delivers the catalogs deep into the mail system, where the USPS benefits by delivering too many catalogs to customers. Then trade journals tell us all about multichannel customer behavior, funded by the profits the vendor community get from matchback analyses.

It looks to me like the entire catalog ecosystem benefits from flawed multichannel matchback analyses. The only parties who don't benefit are customers, who may not want the catalogs, and catalogers who over-mail catalogs, causing harm to the profit and loss statement.

This is why I've been told to stop talking, to "get on the multichannel bandwagon". This is why I try hard to freely share information with catalogers and multichannel retailers.

Three Types Of Catalog Buyers, And Profitability

Catalogers face big challenges when evaluating the profit of catalog mailings. Given that matchback analyses have long over-stated profitability (at the benefit of the vendors providing these analyses, or the list/co-op industry, folks who depend upon inflated catalog results for improved financial success), we've trained a generation of catalog and online marketing experts to evaluate catalog profitability in a suspect manner.

Some catalogers are studying profitability by evaluating quarterly contact strategy tests. These catalogers purposely choose to not mail segments of customers for three months at a time. At the end of the test period, the difference in performance between the mailed and control group is evaluated.

There are three types of catalog buyers that are frequently evaluated.

First, let's evaluate customers who only shop via telephone. These customers are the easiest to measure, because they seldom buy online, meaning our old-school analytical techniques are still effective.

Quarterly Test Results
Audience = Telephone - Only Buyers
	Phone	Online	Stores	Total
Mailed Group	$15.00	$2.00	$2.00	$19.00
Not Mailed Group	$0.00	$1.00	$1.50	$2.50
Increment	$15.00	$1.00	$0.50	$16.50
Demand			$16.50
Net Sales	80.0%		$13.20
Gross Margin	50.0%		$6.60
Less Book Cost			$3.00
Less Pick/Pack/Ship	11.0%		$1.45
Variable Profit			$2.15

This analysis is straightforward. The mailing strategy generated $16.50 demand and $2.15 profit per customer. Matchback analyses are typically accurate among this audience, due to limited spend in the online or retail channels. As long as online/retail spend is minimal, matchback analyses are accurate.


The second segment of customers provide more of a challenge. In the past twelve months, these customers shopped via telephone, and shopped via the internet. Here is what the results can look like within this audience/segment.

Quarterly Test Results
Audience = Telephone + Online Buyers
	Phone	Online	Stores	Total
Mailed Group	$7.00	$8.00	$2.00	$17.00
Not Mailed Group	$0.00	$4.00	$1.50	$5.50
Increment	$7.00	$4.00	$0.50	$11.50
Demand			$11.50
Net Sales	80.0%		$9.20
Gross Margin	50.0%		$4.60
Less Book Cost			$3.00
Less Pick/Pack/Ship	11.0%		$1.01
Variable Profit			$0.59

This is where matchback algorithms begin to fail. The matchback algorithm will take credit for all $8.00 per customer spent online, allocating that revenue to the catalogs that were mailed. Mail/holdout tests tell us the true story, however. Had catalogs not been mailed, $4.00 would have happened online anyway.

Your matchback vendor tells you that you got $7.00 over the phone, and $8.00 online, so all is good! In reality, you got $7.00 over the phone, and $4.00 online --- profit isn't nearly as good.


The third audience includes customers who only shop online. Multichannel pundits strongly believe that catalog mailings drive these customers online. Here's what one might observe, after a quarterly contact strategy test.

Quarterly Test Results
Audience = Online - Only Buyers
	Phone	Online	Stores	Total
Mailed Group	$2.00	$13.00	$2.00	$17.00
Not Mailed Group	$0.00	$9.00	$1.50	$10.50
Increment	$2.00	$4.00	$0.50	$6.50
Demand			$6.50
Net Sales	80.0%		$5.20
Gross Margin	50.0%		$2.60
Less Book Cost			$3.00
Less Pick/Pack/Ship	11.0%		$0.57
Variable Profit			($0.97)

This audience is treated incorrectly by matchback algorithms. Your matchback vendor will tell you that you got $2.00 via the phone, and $13.00 online, yielding $15.00 total. Your matchback vendor will tell you that this is good!!

However, your mail/holdout test results tell you something different. Had you not mailed catalogs, you still would have gotten $9.00 of the $13.00 online. Therefore, when you run the incremental profitability calculation, you find that catalog marketing is unprofitable in this audience.

The reality is that natural search, paid search, e-mail marketing, affiliate marketing, portal advertising, shopping comparison marketing, word-of-mouth, and brand recognition all contribute to the $9.00 of volume you achieve if you don't mail catalogs to this customer.


This type of analysis is sorely missing in modern catalog planning. Some matchback vendors understand these issues, and genuinely try to help us. Sometimes, the thought leadership simply isn't there --- and it is costing catalog marketers millions of dollars of profit.

My level of frustration on this topic continues to grow. Recently, I was told by a vendor-based industry leader to stop talking, and "get on the multichannel bandwagon".

I have no problem with multichannel marketing. I do have problems with industry leaders that mislead (maybe not purposely) catalogers in a way that harms catalogers, but helps the very vendor industry that depends upon catalogers for success.

United Kingdom: Catalog Driving 70-80% Of Online Orders?

Jim Fulton points us to this article, authored by the Catalogue Exchange in the UK. The article suggests that catalogers, using traditional matchback algorithms, are driving 70% to 80% of their online orders via catalog mailings.

The UK online market is a lot like the US online market five years ago. We read a lot of these articles in the US five years ago.

Early in the maturity of the online channel, catalogs are going to be the primary driver of online volume. Most likely, the study is directionally accurate.

Things get very interesting once the customer is trained to shop online, and decides she no longer needs the catalog to shop online. For some brands, this change in behavior never happens. For others, this happens very quickly. I lived through this change at Nordstrom --- folks with a traditional catalog mindset, very bright folks, don't understand the phenomenon when I describe it to them.

If you're reading this in the UK, here's a checklist for you:

• This study was published by a group of individuals with a vested interest in promoting catalog marketing. Just keep that fact in mind.

• Multichannel Forensics represent another way of understanding the long-term impact of catalog advertising, across all channels.
  

• Matchback analyses are highly biased. Want to learn how biased they are?
  
• Identify who you were going to mail your next catalog to.
  
• Take a sample of the audience.
  
• Split that sample in half.
  
• One half receives your next catalog.
  
• One half does not receive the next catalog.
• After twelve weeks, do a matchback analysis on the sample that received the catalog.
• After twelve weeks, do a matchback analysis on the sample that did not receive a catalog --- just don't tell your matchback vendor that they didn't receive the catalog.
• In this group, zero (0) customers ordered online because of the catalog --- they didn't receive the catalog!! If you matchback vendor tells you that online orders happened because the catalog was mailed, you've identified the magnitude of your matchback bias.
• Subtract the difference between matchback results in the mailed and not-mailed group. This is the true number of orders your catalog drove to the online channel.

Matchback Analysis Challenge

Catalogers --- here's what I'd like for you to do on your next catalog.

Step 1: Randomly select 10,000 customers from your mailing.

Step 2: Divide these customers into two groups.

Step 3: In the first group, code these customers as "mailed". Send these customers a catalog.

Step 4: In the second group, code them in your promotional history as "not mailed". Do not send these customers a catalog.

Step 5: After these catalogs have been mailed and results have been measured, send each group of customers to your favorite matchback vendor, be it Experian or Abacus or anyone else.

Step 6: DO NOT TELL EXPERIAN OR ABACUS THE DIFFERENCE BETWEEN THE TWO GROUPS. Simply perform the matchback analysis on each group.

Step 7: Carefully analyze the results in each group. Remember, the second group DID NOT RECEIVE A CATALOG. Therefore, every order that Experian or Abacus attributes to a catalog in this group represents an "overstatement", a "matchback bias" in web orders that are attributed to catalog mailings. Remember, these customers DID NOT GET A CATALOG, you're just communicating to your matchback vendor that they got a catalog, so that you can measure the "matchback bias" that exists in all of these analyses.

After you've done this little experiment, share the results with my audience. I'll be happy to publish what you've found, regarding "matchback bias". I think you'll be surprised by what you learn from this experiment. I think you'll see that you are over-stating the importance of catalog mailings at driving web sales.