Ensure good customer experience while price testing

So you’ve decided that you want to AB test your prices on your Shopify store - congratulations! You’re on your way to making more profit. But - how do you do it while keeping your customers happy? It’s a question we get all the time at Intelligems and we wanted to share some best practices.

Some of the questions we often get asked are:

• Will customers see different prices during my AB test?

• Will customers realize that I am AB testing my prices?

• Will my returning customers get made at me when I change prices?

The good news is that all of these can be solved! We have run hundreds of price tests - touching more than 25 million shoppers - so know how it’s done.

There are 5 best practices for making sure that you run a smooth price AB test that keeps your customers happy

1. Keep your price AB tests relatively short, and iterate quickly

We recommend that price tests last 6 weeks or fewer. The longer an AB test runs, the harder it is to keep it “clean” and make sure everyone stays in the same group.

For example, if someone gets a new phone/computer, or clears their cache fully, they may get “reset” into a new group and see a new price.

Besides being a customer experience risk, your data will also be biased and stale over time from your price test.

2. Keep your Customer Experience team in the loop about testing!

If your customers do have any feedback about prices, your CX team is going to be the first to hear about it. So keep your CX team fully informed about what price AB testing is happening. This will help them smoothly answer any questions.

For example, if someone reaches out saying that they’re seeing a higher price than they saw previously, the CX person can have a code on hand to honor that lower price - turning an “iffy” situation into a moment of customer delight

3. Check all the places - including off the site - where you may mention price!

When you run a price AB test, there are a few spots that you know you will need to change the price - e.g., on your Shopify PDP, on your collections pages. But what about places not on your Shopify site? Make sure you identify these and plan accordingly.

Some common places where price is shown are:

• Automated email campaigns (e.g., abandoned cart emails from Klaviyo)

• Affiliate marketing (e.g., blog posts talking about your product)

• Ads that have price in the creative
  
  

For Product Listing Ads (e.g., Google Shopping) - most good price AB testing tools will have a way to automatically update prices there!

4. Consider excluding certain traffic from your tests

Some Shopify Price AB Testing tools allow you to exclude certain segments of traffic from your tests. This can be a powerful tool to fine-tune your price AB tests to deliver excellent customer experience.

A couple examples of how excluding traffic in a price AB test can be helpful:

• You decide to exclude any direct/organic traffic from your price AB test to avoid impacting returning customers. So only “cold” paid traffic is in the test

• You decide to exclude traffic coming from particular affiliate links, who may have already been anchored on a particular price point

5. Pick the right technology provider and do a high-quality integration

There are many tools to do price AB testing on the Shopify App Store, and some brands even build their own testing tools.

Running a price test is very complex - prices can appear all over your store (and outside of it), and you need to make sure that these are all integrated with in order to avoid any customer experience risk from a Price AB Test. Choosing an easier/cheaper option up front may cost you in the long run.

Additionally, you should make sure that your technology provider has the features to implement the best practices above.

We are biased, but Intelligems fits the bill for all of these! If you’re interested in talking more about best practices here - and how to safely launch price AB tests - reach out on our site or find us on the Shopify app store!