Industry: Banking
This year we've done a few A/B tests of just subject lines for B2B e-mail campaigns. The "standard" way we split the campaign audience was by halving the dataset equally. The first half of the e-mail addresses receive Subject Line A, the latter half receive Subject Line B. This got me to thinking that maybe customers will read just about anything we send them, and if our content is unclear they'll hit reply and/or call up their salesperson. And, that the population subset where campaign readership was higher might also reflect how clean the dataset was.
The customer population used for all three broadcasts is the same dataset.
March
E-mail Focus: Product
Test A: NINA/No Doc - $750K to 80% LTV
Test B: NINA/No Doc - Non-Traditional Solutions
May
E-mail Focus: Product
Test A: New 1-Month LIBOR for the Building Season
Test B: New Construction Solutions
July
E-mail Focus: Technology Enhancement
Test A: Private Label Marketing A Click Away
Test B: Let Us Market for You
The test results are as follows:
Month, A/B, qty e-mails sent, 1-wk opens, 1-wk open rate, 3-wk opens, 3-wk open rate
March A, 21019, 9809, 47%, 10450, 50%
March B, 21022, 9286, 44%, 9840, 47%
May A, 22024, 7766, 35%, 8152, 37%
May B, 22028, 9563, 43%, 10153, 46%
July A, 22600, 8698, 38%, 9580, 42%
July B, 22604, 9468, 42%, 10330, 46%
You might infer that a 3% difference in open rate is not statistically relevant. Even in market research where our audience size was much smaller, at least 8-10% is noteworthy, but not 3%. Not for this population size. Maybe if we did consumer marketing and broadcasted to millions of addresses per month would this be more relevant, but I digress. July is about the same with 4% not being that significant either.
So what happened with the May campaign? Perhaps it isn't data related at all. Spring to Fall is when a lot of home construction and remodeling goes on. A product that has good rates for just its first month isn't as appealing as say, a more general umbrella of solutions offered to a customer.
One thought in the aftermath of the May campaign was that dataset B responded more favorably to campaign broadcasts. Sure, I might be inclined to believe that. However, for the July A/B campaign, the second half of the population was sent the A campaign, and the first half were sent the B campaign. As the results show, the audience didn't care one way or the other.
Analysis: Inconclusive. The only good way to measure e-mail campaign success is to tie the data to quantitative metrics, like how much loan volume resulted from a broadcasted campaign, how much of a media mix was used to promoted a particular product, how many calls were generated by this campaign. ROI isn't necessarily a good metric anymore especially when your company broadcasts in-house. It costs us almost nothing to broacast, except the time spent by the business units putting the campaign content together, myself formatting the content into html/aol campaign templates and pulling data for broadcasting.
So, is it worth the effort? No. But that isn't going to keep us from testing this process with future campaigns.
Candied Woes
Caution is needed when ordering candy online. Our creative services dept had ordered watermelon lollipops. On the backside of the wrapper, there were printed black seeds. On the front was a sticker which read "Compliments of.."
After receiving an e-mail notice that read: "Don't eat the candy in the sales care package, we've encountered a problem with it," an uproar of laughter came from our creative services area.
Ants, it seemed, cleverly disguised themselves as watermelon seeds and managed to adhere themselves to the candy on the inside of the wrapper. Because they were behind the pre-printed watermelon seeds no one noticed they were there.
In some countries this would be seen as a delicacy.
The vendor we used is called Lollies Galore. Candied ants might be good for say.. Halloween, but probably not for customers or salespeople.
After receiving an e-mail notice that read: "Don't eat the candy in the sales care package, we've encountered a problem with it," an uproar of laughter came from our creative services area.
Ants, it seemed, cleverly disguised themselves as watermelon seeds and managed to adhere themselves to the candy on the inside of the wrapper. Because they were behind the pre-printed watermelon seeds no one noticed they were there.
In some countries this would be seen as a delicacy.
The vendor we used is called Lollies Galore. Candied ants might be good for say.. Halloween, but probably not for customers or salespeople.
Specifications
A request to send a survey to customers in Northern California came by my desk the other day. The business unit originating this request failed to specify criteria with respect to which area or region defines what a northern California customer is. A popular marketing way to do this is by identifying a region by zip code. California's zip code range is 90000-96199. The messiest way is to identify this by a list of cities or MSAs (metropolitan statistical areas). What you, as the business unit, shouldn't do is make broad sweeping suggestions as to what you want without knowing what it is that you want.
Disneyland uses the zip code range of 90000-92999 for SoCal resident ticket promos. Another website, www.us.endress.com, suggests that the northern California zip code range is 93600-96199, and breaks southern California into two groups, "90000 to 92999" and "93000 to 93599," however that's just how their sales territories are split for California.
None of these will really do it for this campaign broadcast since what constitutes as northern California is both industry and company-specific. Ventura starts at 93000 and Santa Barbara starts at 93101, both of which are within an hour's drive to Los Angeles. What is good for Endress may not be good for this mortgage company. What's still a bit scary for me is that my inner consultant is telling me to pull a strategy from the air wafting above my head and implement it as a marketing standard for a moving-foward point, since no standard exists for this type of broadcast.
So, what's my resolution? It looks like this state will be split into northern (93600-96199) and southern (90000-93599) California. Unfortunately this is only a bandage solution for what probably wouldn't be aligned with the segmentation goals for the year. Most often when it comes to data segmentation, internal and external clients simply don't know (insert how, what, who, etc). But they're asking you the subject matter expert, the consultant (or in my case, the e-mail marketing manager).
Disneyland uses the zip code range of 90000-92999 for SoCal resident ticket promos. Another website, www.us.endress.com, suggests that the northern California zip code range is 93600-96199, and breaks southern California into two groups, "90000 to 92999" and "93000 to 93599," however that's just how their sales territories are split for California.
None of these will really do it for this campaign broadcast since what constitutes as northern California is both industry and company-specific. Ventura starts at 93000 and Santa Barbara starts at 93101, both of which are within an hour's drive to Los Angeles. What is good for Endress may not be good for this mortgage company. What's still a bit scary for me is that my inner consultant is telling me to pull a strategy from the air wafting above my head and implement it as a marketing standard for a moving-foward point, since no standard exists for this type of broadcast.
So, what's my resolution? It looks like this state will be split into northern (93600-96199) and southern (90000-93599) California. Unfortunately this is only a bandage solution for what probably wouldn't be aligned with the segmentation goals for the year. Most often when it comes to data segmentation, internal and external clients simply don't know (insert how, what, who, etc). But they're asking you the subject matter expert, the consultant (or in my case, the e-mail marketing manager).
A/B Testing for E-mail Campaigns
A/B testing, also called an A/B split, is the easiest method of testing elements within your emails or on your Web site. The target audience is divided into two groups. One group is broadcasted the original version of whatever you are testing, and the second group sees an alternative version, where only one element has changed. Then, the results are tracked. With e-mail marketing, the following components are usually tested: bonus gifts, coupons, P.S. messages, guarantees, opening sentence image, closing sentence image, from-field, calls to action, opening greetings, type styles, layout elements, graphic images, etc. This form of testing is used by both business and consumer-driven campaigns.
My company did its first A/B e-mail campaign test last week. The only element that changed was the e-mail subject line. We used:
Test A: NINA/No Doc - Non-Traditional Solutions
Test B: NINA/No Doc - $750K to 80% LTV
On average, our audience open rate for product e-mails is about 45%, it's not that great and could use some improvement. The distribution list was split 50/50. After 5 calendar days, there's no discernable difference between either subject line and the combined open rate is 42%. Having not worked in the mortgage industry before, much of the e-mail content that we broadcast doesn't make much sense and is so banking specific that I hope our customers are getting value from it.
My company did its first A/B e-mail campaign test last week. The only element that changed was the e-mail subject line. We used:
Test A: NINA/No Doc - Non-Traditional Solutions
Test B: NINA/No Doc - $750K to 80% LTV
On average, our audience open rate for product e-mails is about 45%, it's not that great and could use some improvement. The distribution list was split 50/50. After 5 calendar days, there's no discernable difference between either subject line and the combined open rate is 42%. Having not worked in the mortgage industry before, much of the e-mail content that we broadcast doesn't make much sense and is so banking specific that I hope our customers are getting value from it.
Email Authentication
Today's web surfing takes me to investigate email authentication and its implementation costs. The hooplah that started this inquiry was from a press article about Goodmail partnering with AOL and Yahoo! to essentially get rid of the free whitelist system and replace it with a fee-based system for commercial senders. The concern for a publicly traded bank is whether or not its competitors will adopt such an authentication system to have their emails handled on a preferential basis by a consumer because it contains a "trust" tag.
What does email authentication do? Well, for starters it tries to manipulate how a commercial email is handled by an email service provider. The perception for the consumer is exactly like that of a 1st Class postage stamp versus a pre-sorted indicia, the latter of which almost entirely is for bulk postal mail, like pieces addressed to "Dear Head of the Household" or to "Current Resident." Here's Wikipedia's definition.
Goodmail is new to email authentication and has only been in existence since June 2003; not nearly enough time for it to have developed relationships. Goodmail's online documentation infers that by using their paid token system, it guarantees delivery of a commercial sender's email without blocking html, pictures, or embedded hyperlinks, to a consumer's email inbox. Click here to view their process.
There is a lot of unseen resources consumed by using or considering to use an email authentication service. Goodmail, for example, is a fee-driven company. The more fees, the more revenues. Let's just look at a few of the ones from Goodmail. There's a $399+ signup fee, an accreditation fee, token purchase fees which can rise by 25% if you ever show up on their watchlist as a probationary company, and per-email fees based on email volume.
I asked Goodmail to send me case studies of companies in the banking industry who are currently using their service and a list of references; the basics of consideration for any vendor offering. So far, no response yet.
Best practices are still the way to go, and the FTC has guidelines for business to comply with the CAN-SPAM Act of 2003.
What does email authentication do? Well, for starters it tries to manipulate how a commercial email is handled by an email service provider. The perception for the consumer is exactly like that of a 1st Class postage stamp versus a pre-sorted indicia, the latter of which almost entirely is for bulk postal mail, like pieces addressed to "Dear Head of the Household" or to "Current Resident." Here's Wikipedia's definition.
Goodmail is new to email authentication and has only been in existence since June 2003; not nearly enough time for it to have developed relationships. Goodmail's online documentation infers that by using their paid token system, it guarantees delivery of a commercial sender's email without blocking html, pictures, or embedded hyperlinks, to a consumer's email inbox. Click here to view their process.
There is a lot of unseen resources consumed by using or considering to use an email authentication service. Goodmail, for example, is a fee-driven company. The more fees, the more revenues. Let's just look at a few of the ones from Goodmail. There's a $399+ signup fee, an accreditation fee, token purchase fees which can rise by 25% if you ever show up on their watchlist as a probationary company, and per-email fees based on email volume.
I asked Goodmail to send me case studies of companies in the banking industry who are currently using their service and a list of references; the basics of consideration for any vendor offering. So far, no response yet.
Best practices are still the way to go, and the FTC has guidelines for business to comply with the CAN-SPAM Act of 2003.
In-flight Advertising
On a recent trip from Seattle to LAX, I noticed two notable avenues to the way passengers are exposed to advertising. The first was listening to a co-branded message between Alaska airlines and VISA credit cards, broadcasted by the pilot. It didn't seem to evoke much interest among the coach passengers. I'd say that it's more like we were too tired to care about another credit card offer. The second was a $20 coupon of a restaurant I'd never heard of which was handed out by a stewardess.
It's not surprising though. The Nov 2005 press release from UK-based Ink Publishing (ink-publishing.com) notes that "global inflight advertising revenues are predicted to grow from US $685 million in 2004 to US $832 million this year and will exceed US $1,010 million in 2006, with increased general advertising spending plus the opening of niche markets by point-to-point airlines taking credit for the growth." These sales figures are based on their own portfolio data of inflight titles they publish. You can imagine how tremendously large this market really is.
What is surprising is how late-to-market some airlines are with capturing their share of this revenue stream.
It's not surprising though. The Nov 2005 press release from UK-based Ink Publishing (ink-publishing.com) notes that "global inflight advertising revenues are predicted to grow from US $685 million in 2004 to US $832 million this year and will exceed US $1,010 million in 2006, with increased general advertising spending plus the opening of niche markets by point-to-point airlines taking credit for the growth." These sales figures are based on their own portfolio data of inflight titles they publish. You can imagine how tremendously large this market really is.
What is surprising is how late-to-market some airlines are with capturing their share of this revenue stream.
It is what it is
Yes, I am opinionated, especially about marketing and a lot of what goes on in this blog seems to look down upon many practices within the industry. I call it how I see it, with both an agency-side and client-side perspective. I recommend any marketing student fresh out of college to get experience from both sides of the fence. It really helps with perception and anticipation of client needs. If you ever come across a marketer without an opinion, turn around and walk away quickly.
Here're my new year's wishes..
I wish that people would be more honest and open about how they interact with other people, both on a social and business level.
I wish that corporate nirvana wasn't so boring. I recently left a job where I was in total control of the direction and scope of market strategy for the company. I reported to the CEO, and occassionaly to the Sr VP of Sales and Marketing for sales related issues. I had one direct report. I created and managed marketing campaigns. I developed and launched two new marketing services. I revamped the sales lead generation system to be entirely paperless. I negotiated vendor prices for data acquisition projects. I did legal compliance for telemarketing data and email marketing, eventhough we had inhouse corporate counsel. I built a website to generate ad revenue for the companies partnered and doing business with ours. I did a lot of things. But, I got bored and frustrated that I couldn't do more. How could I do more? Well, for one I would have liked a set, quantifiable, spendable budget instead of having to pick and choose between projects that I thought would be good to enhance the brand image of the company instead of ones that would gain the CEO instant gratification on short-term benefits.
Yes. I have issues. Everyone has issues. I thrive on making the world better, making marketing a legitimate science, making health defying desserts, and ranting on about what can be done to make this a better place.
My holy grail of marketing is to learn and know everything there is to know about marketing in each of the four marketing segments (business, consumer, non-profit, and government) and become more aware of industry trends affected by marketing. The ultimate test of a marketer is not the job itself, but what you do with that knowledge. Anybody can work for a living. That's no sweat. Validation comes from the outside.. from writing journal articles that aren't meant as sales pitches for your own company, from teaching college courses, from revitalizing the way people think about marketing. The best thing of all is that learning is a continuous process and marketing is always evolving.
Here're my new year's wishes..
I wish that people would be more honest and open about how they interact with other people, both on a social and business level.
I wish that corporate nirvana wasn't so boring. I recently left a job where I was in total control of the direction and scope of market strategy for the company. I reported to the CEO, and occassionaly to the Sr VP of Sales and Marketing for sales related issues. I had one direct report. I created and managed marketing campaigns. I developed and launched two new marketing services. I revamped the sales lead generation system to be entirely paperless. I negotiated vendor prices for data acquisition projects. I did legal compliance for telemarketing data and email marketing, eventhough we had inhouse corporate counsel. I built a website to generate ad revenue for the companies partnered and doing business with ours. I did a lot of things. But, I got bored and frustrated that I couldn't do more. How could I do more? Well, for one I would have liked a set, quantifiable, spendable budget instead of having to pick and choose between projects that I thought would be good to enhance the brand image of the company instead of ones that would gain the CEO instant gratification on short-term benefits.
Yes. I have issues. Everyone has issues. I thrive on making the world better, making marketing a legitimate science, making health defying desserts, and ranting on about what can be done to make this a better place.
My holy grail of marketing is to learn and know everything there is to know about marketing in each of the four marketing segments (business, consumer, non-profit, and government) and become more aware of industry trends affected by marketing. The ultimate test of a marketer is not the job itself, but what you do with that knowledge. Anybody can work for a living. That's no sweat. Validation comes from the outside.. from writing journal articles that aren't meant as sales pitches for your own company, from teaching college courses, from revitalizing the way people think about marketing. The best thing of all is that learning is a continuous process and marketing is always evolving.
Subscribe to:
Posts (Atom)