The Power of A/B Testing — Online and Offline

In the world of outreach and organizing, there has been a growing emphasis on list size. That is to say, the leadership of many organizations has become focused on the total number of followers, e-mail subscribers, web visitors and direct mail recipients they have racked up. Often, success is defined by the ability to grow that number.

While list growth is a good thing, equal (if not more) emphasis should be given to the ability to engage existing subscribers, visitors and supporters. After all, the cost of outreach is not fixed, and often dependent on list size — whether it’s hiring more staff to manage it, paying postage to mail letters or increasing server capacity to handle more traffic. A highly engaged list of a few thousand is almost always more effective than a stale list ten times that size.

One tactic that may lead to increased engagement is “A/B testing.”

A/B testing involves presenting or sending two or more differing messages to a statistically significant group of supporters, then measuring the results. The differences in the messages could be as small as a single image or subject line or as sweeping as a completely different narrative. Once the most successful message has been determined, it is then circulated to the remainder of your list, thereby increasing the impact of a single message.

A/B testing has been used successfully by everyone from major companies like Amazon.com to large political campaigns and non-profit organizations.

In recent posts, we have discussed the various types of technology consulting offered by Intesa. As a technologist (and professional geek), it’s easy to put things in boxes, which is why it bears mentioning that A/B testing can provide gains not only to your online organizing & outreach efforts (web and e-mail), but also your offline programs (including direct mail, phone and other efforts).

Something as simple as the phrasing of a call to action can actually have drastic impacts on potential customers and supporters — improving not only the percentage who convert, but the amount that each person purchases or contributes.

The best part is that your office doesn’t need a team of data scientists and support staff to implement A/B testing — Intesa can help craft, test and implement such messaging, so your team actually has more resources. We can even train your staff to develop a data-based approach to messaging, so your organization can use this and other tactics moving forward.

Although it isn’t difficult or complex to implement, many companies and organizations overlook data-driven tactics like these, giving you and your team an opportunity to get the upper hand and make your messages stand out in an ever-increasingly crowded space.

Leave a Reply

Your email address will not be published. Required fields are marked *