Kurt Medina, a Philadelphia-based consultant, recently told a story of a client who asked his opinion on a format and three creative executions. Kurt immediately suggested that they test to see what would work best. Then he further suggested that testing the offer would ultimately be more valuable than testing the creative.
Kurt’s client objected, saying, “No, we don’t need to test. That’s why we hired you – the expert. We want you to tell us which package will win.”
Naturally, Kurt replied, “Because I am the expert, I will tell you I don’t have the slightest idea! That’s why we test.” But his client wasn’t interested in waiting for test results and Kurt eventually lost the business.
This story – and others I’ve heard just like it – force me to ask, “When did testing become a bad thing?”
Direct Marketers are lucky. Because each and every one of our sales is the result of a two-way conversation with individual consumers, we know exactly who bought…when they bought…and which piece of advertising got them to buy. We never have to say, like retailer John Wanamaker, “I know half my advertising doesn’t work. I just don’t know which half.”
Instead, we know exactly which package we mail, which ad we run, or DRTV spot we air works. Furthermore we can test every aspect of our advertising to hone our winning advertising until it is a veritable money-making machine.
So what gives? Why are direct marketers singing along with the Grateful Dead, “Test me, test me. Why don’t you arrest me?”
Perhaps it’s because so many have forgotten the basic rules of testing, which to my mind are:
1. Test the big things. Before testing to see if a pink envelope will beat a blue, test the things that can really move the needle. These include:
• The list or media
• The offer
• The timing
• The format
• The copy platform
• The elements of your product or service
2. Test one thing at a time. Sounds simple doesn’t it? But I’ve seen far too many folks test the offer, the list and the size of the order form in a single test cell and then wonder which one actually accounted for the rise – or fall – in response.
3. Leave the control alone. You need something reliable to test against, so resist the temptation to “tweak” your control while you’re testing a new creative, a new offer or a new format against it. Once you’ve made a change to your control, you’ve invalidated your test.
4. Make sure your sample size is large enough and random enough. Direct marketing works on the laws of large numbers … so don’t set up five tests when you’re only mailing to 50,000 pieces. And don’t say, “List A and List B are about the same size, so I’ll just mail Creative 1 to the first and Creative 2 to the second.” Always do an A/B split to insure that there is as little variation in your test group as possible.
Or perhaps the reason so many marketers want to pass on testing is because they think they are saving money. How could they have forgotten this axiom – testing pays for itself!
Test a new offer against your control and one of two things will happen.
1. Your new offer wins. Bravo! You now have a license to make more money … and the increase in customers more than paid for this valuable knowledge.
2. Your control offer wins. You now know that your control is still working fine …
and the sales that from your test cell help to cover the cost of testing.
See? Testing is a reliable way of maximizing sales, and increasing ROI while making the sales that keep your business strong and profitable. It’s a true win-win situation.
So why pay an expert to guess at what might work? Instead take the time and make the effort to test … and enjoy the fruits of your labors.
Great post! You really hit the nail on the head as to why direct marketers need to test, test, test.
Oh, the pain of meeting with direct marketers who don’t test. A few years ago at a direct marketing conference, I chatted with some people from a housewares retailer who used fancy mail order catalogs. I asked how they decided who got what mailings and how often. They laughed – these people knew they should test, but their management insisted on mailing to everyone on their list every time they issued a catalog. As I watched my mail I saw that this company sent me at least 4 catalogs a year, year after year, although I had never made any purchase from them. Of course, that company isn’t in business any more.