#@@#@!![Experiment] How AI is Changing the Way We Optimize at Unbounce

Posted on Posted in Blog

When I very first encountered A/B testing, We immediately wanted to become the type of marketer who tested everything . The idea sounded fun to me. Like being a mad scientist operating experiments to prove whenever my work was really “working. ”

Turns out although, there’s always a long list of other things to do first… blog posts to create, campaigns to launch, and don’t get me began on the meetings! I’m not alone in this, either. A lot of internet marketers are just too darned active to follow up and enhance the stuff they’ve currently shipped. According to HubSpot, only 17% of marketers use landing page A/B tests to enhance conversion rates.

My to-do list
A small glimpse of my ever-growing to-do list. Ain’t nobody got time designed for A/B tests.

Sure, running a split test with one or two variants always sounds easy enough. But once you take a closer look at the process, you realize just how complex it can actually be. You need to make sure you have…  

  • The right length and sample size.
  • Taken into account any exterior factors or validity risks.
  • Learned how you can interpret the results correctly, as well.

But—while there will often be a time and place for A/B testing— there’s also now an easier and faster way for marketers to enhance . Smart Traffic is really a new Unbounce tool that will uses the power of AI and machine learning to get you more conversions. Every day, more marketers are using Smart Traffic to “automagically” optimize their landing pages. But whenever we launch anything new, we love to test it out for ourselves to understand alongside you (and keep you up to speed on what to try next).

Here’s what I learned after taking Smart Traffic to get a test drive myself…

Shifting Your Mindset to Optimize along with AI

I know many marketing experts are (perhaps) skeptical with regards to promises of machine understanding, artificial intelligence, or magical “easy” buttons that get them better results. But AI is all around us and it is already changing the way we do marketing. Landing page optimization is just one more area of the work where you no longer need to do everything yourself manually.

Smart Traffic augments your marketing abilities and automatically sends people to the landing page variant exactly where they’re most likely to transform (based on how similar web page visitors have converted before). It makes routing decisions faster than any human actually could (thank you, AI magic), and “learns” which page variant is a perfect match up for each different visitor. This ultimately means no more “champion” variants . Instead, you’re free to produce multiple various pages to appeal to various groups of visitors and run ‘em all at once .

This really is very different from A/B testing and honestly—it can feel kinda weird at first. You’ve got to trust in the machine learning to figure out what works best and exactly what doesn’t. Data scientists call this the “black box” problem: data goes in, decisions come out, but you never really have the full understanding of what happened in between.  

How traffic gets allocated in A/B testing vs Smart Traffic
Smart Traffic is essentially different from A/B Testing. You can learn more about how it works here.

For entrepreneurs using Smart Traffic, this means shifting your mindset and starting to think about optimization differently . In contrast to A/B testing, you’re not really looking for those “aha” moments to apply to your next marketing campaign, or a one-size-fits-all “winning” version. Instead, you are looking to discover what works best for different subsets of your audience. This gives you limitless creativity to try out new marketing ideas, makes it easier and less risky for you to improve, and gives you an average transformation lift of 30% when compared with splitting the traffic equally across multiple variants. (Woah. )

My Experiment with Smart Traffic

I know all this mainly because I recently experimented with variant creation myself to better understand this new AI optimization mindset. We created 15 variants across two separate landing pages using Smart Traffic to discover…

  • How easy could it be to optimize with an AI-powered optimization tool?
  • Could I quickly set up the particular tests in Unbounce while still getting those additional to-do’s done?
  • What kind of conversion lift would certainly I see from just a few hrs invested?

I got a little bit of my inspiration from Ms. Frizzle on the Magic School Bus. No, not really her haircut, her catchphrase: “Take chances, make mistakes, get messy! ”

Magic School Bus Gif
Oh, so that’s where she got all of the her good ideas.

Creating 15 Variants in less than Two Hours

The beauty of Wise Traffic is there are simply no limits to how many variants you can create and it automatically starts optimizing in as few as 50 visits. Just hit the “optimize” button and you’re off to the competitions. Could it really be that simple?

My guinea pigs with this experiment would be two current campaigns our marketing group had worked on: the e-commerce lookbook and the SaaS optimization guide. The team acquired created both of these ebook download pages in Unbounce, but we hadn’t been able to return to them and optimize completely in the months since all of us published.

The original landing pages would serve as my control variants. (Click to see the full pages. )

Prior to starting, I consulted with Anna Roginska, Growth Marketer at Unbounce , to get her input about how I should create my variations. She advised:

Anna Roginska, Growth Marketer, Unbounce

You can take the ‘spaghetti at the wall’ technique, where you create a bunch of variants and just leave them to Smart Traffic to see what happens. It’s that ‘set it plus forget it’ mentality. That’s interesting, but when you look at the bowl of spaghetti… There’s plenty of noodles in there. You won’t necessarily get to explain why something is working or not working.

The other approach is to be more strategic and concentrated. I think there’s a huge advantage to going in with a plan. Create maybe only 5 variants and give them every a specific purpose. Then, you can view how they perform and create new iterations for different portions from the audience.

I had two landing web pages to work with, so I thought I’d give both approaches a try. But with only a few hours planned in my calendar to complete all these variants, I needed to proceed fast.

The “Spaghetti in the Wall” Approach to Variant Creation

On the ecommerce lookbook page, I wanted to spend less time preparing and more time creating . Whereas within A/B testing you need a proper test hypothesis and a careful plan for each variant, Sensible Traffic lets you get innovative and try out new suggestions on the fly. Your variants don’t have to be perfect—they just need to be different enough in order to appeal to new audience segments.

This meant I didn’t have to make any kind of hard or fast options about which one element in order to “test” on the landing page. I can create 15 different variants that varied wildly from one another. Some used colours, some had different head lines, some completely changed up the layout of the page.

Some of the variants

This is some thing you just can’t do in a traditional A/B test where you are looking to find a “winner” and understand why it “wins. ” I had to remind me personally I wasn’t looking for that certain variant to rule them all (or for that one version to bring them all and in the darkness bind them). I used to be looking to increase the chance of transformation for every single visitor. Certain pages were going to are more effective for certain audiences, and that has been completely fine .

We wondered, though: how many variants would be a lot of ? Would the machine learning recognize that some of these were not anything special and just stop delivering traffic to them? And how long would it take to get results? With these questions in mind, I actually checked back on the first set of tests one month later…

Changing up the background color

Usually, color A/B medical tests are pretty much a waste of time. You need a lot of information to get accurate results, and most marketers don’t actually finish up learning anything useful in the conclusion. (Because color by itself means nothing, it always depends on the context of the page. )

That being said, we know there is several legitimate color theory plus certain audience segments react better to certain colors than others. So I thought it might be interesting to switch up the background on this landing page to see what would happen. And color myself surprised—these variants are viewing some pretty dramatically different conversion rates:

  • Pink background – 12. 82%
  • Green background – 21. 43%
  • White background – twenty one. 74%
  • Black background – thirty-one. 71%

One particular might start to speculate from these conversion rates that darker backgrounds perform better than the lighter backgrounds. But hold your own horses, that’s thinking about this particular as an A/B test once again. Here’s why Jordan Dawe, Senior Data Science Creator at Unbounce, says you should be cautious about sketching any conclusions from the transformation rates…

Jordan Dawe, Unbounce

Smart Visitors is not sending visitors randomly—it’s trying to get the best traffic to the best variant. So in this case, this doesn’t mean that a dark background will always convert greater than a pink background. You can find likely portions of the viewers going to each color that would be doing worse on others. Here’s what you can consider: the color black is preferred by a portion of the traffic that converts highly.

It’s hard to wring that mindset of looking for a “winner” and trying to figure out “why” something is working. But I had been starting to accept that various portions of the audience might always respond better to different variants—this was just the first time I’d been able to use AI to automatically serve up the very best version.

Making big (and small) changes to the topic

For that next group of variants, I actually switched up the H1 in both small and big methods to see what effect that will have on the conversion rate. In some cases, this meant simply swapping a single adjective (e. g., “jaw-dropping” for “drool-worthy”). In other cases, I went with a completely new line of copy altogether.

Here’s how the variations stacked up against each other:

  • See 27 Sales-Ready E-commerce Landing Pages in Our Supreme Lookbook – 25. 81%
  • See twenty-seven Stunning Ecommerce Landing Web pages in Our Ultimate Lookbook – 25. 93%
  • Get Ready to See 27 Jaw-Dropping Ecommerce Landing Page Examples – 28. 13%
  • Get Serious Inspo just for Supercharging Your Ecomm Sales – 35%
  • See 27 Drool-Worthy Ecommerce Landing Pages within our Ultimate Lookbook – forty percent

Again, every variant yielded a different conversion rate. I wondered if I kept testing different variants of the headlines and discovered one that performed best, is there a chance i deactivate all the other headline versions and just go with the “best” one?  

Here’s exactly how Floss Taylor, Data Expert at Unbounce, responded…

Floss Taylor, Unbounce

Clever Traffic doesn’ t have got champion variants. You do not pick one at the end like you might in an A/B test. Although one variant may appear to be performing poorly, there could be a subset of traffic that it’s ideal for. You’re best leaving it on extensive so it can work its miracle.

Trying out various page layouts and hierarchies

The last set of variants I developed messed with the actual construction and hierarchy of the web page. I wanted to see if moving things around (or eliminating sections entirely) would influence the conversion rate. Here is a sample of some of the experiments…

  • Removing the Topic – 16. 67%
  • Adding a Dual CTA – 21. 95%
  • Relocating the Testimonial Up the Web page – 27. 27%

Nothing too surprising here. And because I had produced so many variants, Smart Traffic was taking longer compared to usual in “Learning Mode” to start giving me a transformation lift. Here’s how Get flossing Taylor explains it…

Floss Taylor, Unbounce

Clever Traffic needs approximately 50 visitors to understand which traffic would perform well for each brand new variant. If you have 15 variants and ~100 visitors each month, you’re going to have a lengthy learning period where Wise Traffic cannot make accurate recommendations. I’d suggest beginning with a lower number of versions, and only adding more once you have adequate traffic.

The “Strategic Marketer” Approach

So tossing spaghetti at the wall turned out to be… messy. (New parents beware. ) For that SaaS optimization guide page, I wanted to be a bit more tactical. And I actually had a leg up for this one, because Anna Roginska, Growth Marketer at Unbounce , had already started using a Smart Traffic experiment with this page four months ago.

Anna had set up the test between two different variants. One had an image of the ecommerce lookbook as the hero graphic on the page, while the other used the image of conversion expert and author Talia Wolf. Anna says she decided on this second version because of research she got seen on how photographs of individuals tend to convert better than products.

Anna Roginska, Growth Marketer, Unbounce

I put Talia beforehand because I knew from all other tests I’ve run and research I’ve done. [Photographs of] people often convert better. I didn’t know if it would work much better in this particular case, but I was able to set up the variant and use Wise Traffic to find out. And it just so happens that the algorithm started sending way more traffic to this variant.

Anna seemed to be onto something, too: the girl variant was converting with nearly double the rate for the large traffic subset . And while I now know we can not consider this a “champion” variant like in an A/B test and learn from the results, we could sum up based on her design to new audience segments.

I developed simple spreadsheet to develop our gameplan. The goal had been to create five new versions of the page that would appeal to different visitors based on their own attributes:

Reducing the word count to focus on mobile and “ready to download” visitors

For inspiration on my first variant, I consulted the 2020 Conversion Benchmark Report. The machine learning insights here recommended that SaaS landing web pages with lower word matters and easier-to-read copy often perform better than their long-winded counterparts.

And while the original version of our download page has been easy enough to read, it did have a long, wordy intro with a lot of additional detail. Could I increase the conversion rate for a portion of our audience if simply focused on the bare essentials? I was ready to kill a few darlings to find out…

  • Original Long-Form Version – 10%
  • Reduced Word Count Version – 21. 43%

It seems there’s a segment in our traffic coming to this page who didn’t need to see everything extra info before these people decided to fill out the form. I speculated that this variant might also perform much better on mobile devices since it would be faster-loading and easier to scroll through. Interesting!

Switching the headline to target different target audience segments

Next, I created an additional four page variants approach the different pain points plus reasons our audience might want to download the guide. (Actually, this is something Talia herself recommends you do in the SaaS optimization guide. ) I switched up the headline duplicate here, as well as some of the helping text underneath to match. After a month, here’s what the conversions look like:

  • Get Talia’s Guide to Optimize – nineteen. 05%
  • You can not Just Build – twenty three. 08%
  • Optimization is a Lot of Work – 24%
  • Not Sure How to Optimize? – 33. 33%

Each variant is serving a different segment of the viewers, by speaking to the particular reason they want to download the guide most (e. gary the gadget guy., maybe they don’t have the time to optimize, or maybe they don’t know how to get started). As Smart Traffic learns more about which variants perform best for which audience segments, we become that much more likely to score a conversion.

What I Learned Running These Smart Traffic Experiments

Smart Traffic absolutely makes optimization easier and faster for marketers who previously never had the time (or experience) to run A/B tests. It took me under two hours to set up and launch these experiments, and we’re already seeing some pretty impressive results just over a month later.

While the ecommerce lookbook page is still optimizing, the SaaS ebook page is showing a 12% lift in conversions compared to evenly splitting traffic among all these variants. And this is after only a month—the algorithm will keep improving to get us even better results over time. (Like a fine wine, or that suspiciously old cheese in my fridge. )

At the same time, I did walk away with a few important lessons learned. If you’re planning to use Smart Traffic to optimize your landing pages, here are some things to keep in mind before you get started:

  • There are no champion variants – Unlike traditional A/B testing, you won’t be able to point to one landing page variant at the end of your test and call it a winner. The machine learning algorithm automatically routes audiences differently based on their individual attributes, which means you have to be cautious when you’re analyzing the results.
  • The more variants you create, the longer you’ll wait – While it can be tempting to throw spaghetti at the wall and create dozens of variants for your landing page, this means you’ll also have to wait longer to see what sticks. Try starting out with three to five variations and take a more strategic approach based on research in your industry. (The 2020 Conversion Benchmark Report is a great place to start for some ideas. )
  • It’s (usually) better to leave low-converting variants active – Because Smart Traffic learns over time and continually improves, you’re typically better off leaving your variants active—even if their conversion rates aren’t all that impressive. The AI takes the risk out of optimization by automatically sending visitors to the page that suits them best. If you turn off variants, you may lose out on some of those conversions altogether.

It can be a lot of fun to get creative with the different page elements and try out new ideas. You just might want to come up with a bit of a plan first and be strategic with your approach. Still, it’s better to experiment and optimize with Smart Traffic (even if you make some mistakes along the way) than to never optimize at all.

(And in case you were worried, yep—I managed to get my to-do list done, too. 😅 )

Spread the love

Leave a Reply

Your email address will not be published. Required fields are marked *