AB Testing
Dec 16, 2024
Optimize your profits with a robust testing roadmap
Optimize your profits with a robust testing roadmap
Drew Marconi
Routine experimentation is the key to driving conversions and profitability. It’s the most effective way to secure a lasting competitive edge — keeping your online store aligned with changing customer preferences and market dynamics.
When it comes to A/B testing, higher conversions often mean better performance. But in practice, traditional conversion rate optimization (CRO) — a testing approach focused on elements like site UI and personalized experiences — rarely sparks profitable growth.
That’s because most CRO efforts prioritize conversions over profitability. When the two don’t move together, your margins take a hit. Say you offered free shipping to entice more customers to check out. You’ll increase conversions, but covering the shipping costs for every purchase will cut into your margins.
To make impactful site changes that lift the bottom line, merchants must gear their testing to this specific goal. In other words — rather than prioritizing content, price, and shipping variants that generate higher conversions, your testing decisions should center on the site elements directly tied to optimizing profit.
Profit optimization — the best-kept secret for growing your business
While increasing conversions and revenue may look promising on the surface, the only way to grow in today’s ecommerce landscape is to drive profitability. To do this, brands need to look beyond the instant gratification of CRO by structuring their experiments around their long-term goals.
Profit optimization is the only testing approach that doesn’t leave your net earnings to chance. It maintains the balance across conversions, revenue, and margin, so store adjustments consistently bolster profitability regardless of daily business costs.
And it works — over 95% of brands find new, more profitable prices after three price tests using this strategy.
Use Case: Betting on profit optimization for customer-centric site changes that grow margin
Instead of speculating about the most effective pricing and offers for their shoppers, Fulton, a brand producing sustainable insoles, used customer preferences to guide their business decisions.
Optimizing for profitability showed which site elements simultaneously resonated with shoppers and increased their margins. As a result, all of their critical metrics skyrocketed:
255% increase in primary landing page profit per visitor
245% increase in primary landing page revenue per visitor
220% increase in average landing page conversion rate
Fulton’s success wasn’t a lucky coincidence. Others can achieve similar results, starting with the same first step — building a testing roadmap to activate the key profit optimization levers on their site.
Covering the basics: What is a testing roadmap?
A testing roadmap is a strategic plan outlining the sequence and structure of your profit-focused experiments. While there are many profit optimization levers you can pull across your store, a well-designed roadmap will start with the most impactful options:
🏷 Base pricing
🛒 AOV builders
🎁 Promotions
👥 New customer activation / reactivation
📦 Inventory management
🛠️ Personalizations
Every effective roadmap includes your hypotheses, ideal outcomes, and a general timeline for testing each lever. Defining these elements upfront steers your optimization efforts in the right direction to meet your brand’s customer needs and growth objectives.
Use Case: Building a testing roadmap for a profitable Black Friday campaign
Testing roadmaps are essential co-pilots in optimizing your store ahead of a holiday sprint and securing lasting results, such as increased conversions and profitability.
Momofuku is a prime example. To maximize margins for their upcoming campaign, the contemporary noodle and seasoning brand developed a Black Friday testing plan to determine whether a gift with purchase, free shipping, or a volume discount had the greatest impact.
With their roadmap as their compass, Momofuku used Intelligems Personalizations to:
Set up tailored site experiences for every offer
Add links for each experience to the corresponding promotional emails sent with Klaviyo
Display the specific offers to site visitors who arrive via their email links
Since there’s a suitable lever for every unique objective, you, too, can replicate Momofuku’s strategy for optimal outcomes. As long as — like Momofuku — you let your business goals shape your roadmap and guide you to the right profit levers.
Looking to cement brand loyalty? Setting prices that resonate with customers demonstrates how you value their voice (without sacrificing profit).
Trying to clear slow-moving items from your inventory? AOV builders like bundles and volume discounts incentivize larger purchases and profitably remove excess stock.
Expanding into a new product category? Offering new items as a free gift with purchase helps indicate which options appeal to customers before launch.
As you assemble your roadmap, don’t forget to allocate sufficient time to each test for accurate insights. Remember: To detect a 10% conversion change with 90% confidence, you should keep a test running until you get at least 300 orders from each testing group.
As your results roll in, you can not only modify certain experiments, but introduce new levers (and remove others) — keeping your profit optimization plan flexible and your decisions well-informed.
Building a robust testing roadmap: Pricing, AOV, and Promos
A well-structured roadmap gives you the flexibility to tackle each profit lever at your own pace — rather than rushing through a cookie-cutter testing plan. And to do so for maximum profitability, you need to prioritize your testing elements by impact and complexity.
Between lever trends, best practices, and site elements, a lot goes into creating an actionable roadmap — so we’ve divided this entire process into two actionable guides.
Here, in Part I, we’ll deep dive into the most effective strategies for activating the first three levers on your website. Part II will break down New Customer Activation / Reactivation, Inventory Management, and Personalizations.
Ready for Part II? Access the next three profit levers here.
🏷 Nail your base pricing
Your base prices — the prices you offer outside of promotional periods — have the most direct and powerful effect on revenue.
But decreasing them to bring more site visitors is often not enough to actually boost profitability.
Why? Because the impact your prices have on store performance is much more complicated.
Attractive prices invite more customers — but lower your margins
Many merchants cut prices to draw additional store traffic and sway more customers to purchase. While you generate high surface-level results, the decrease in prices comes at a cost:
Order quantity and conversion rates increase
AOV and profit per order drop
On the other hand, imagine you increased prices. Fewer customers will convert, but the ones that do will drive more revenue without any additional costs, leading to higher profit per order.
In the end, your overall profitability comes down to these two questions:
Will your price cut generate enough orders to make up for the smaller profit you’re making on each sale?
Will the extra profit per order from your price increase make up for lost orders?
To avoid dips in your AOV while securing bottom-line growth, A/B test different prices to identify the ideal — and most compelling — base price for you and your customers.
Finding this inflection point for your first profit optimization lever will prepare you to do the same for other, more complex elements in your roadmap — like your AOV builders and promotions.
Use these best practices to derive maximum value from your pricing.
Gradually narrow the scope of your experiments to ensure data-backed pricing
To pick the right price, you first need to know what to test. Before relying on assumptions alone, follow these steps to create granular, informed experiments:
Run a sitewide test — typically a five- or ten-percent decrease against the same increase. So, if the regular price for your handcrafted mugs is typically about $20, you’d be testing this price against $18 (a 10% cut) and $22 (a 10% increase).
Investigate your results to spot product segments with similar performance. For example, mugs with a Christmas theme might perform better at the lower price since customers are looking for budget-friendly holiday gifts.
Further test within each segment to identify pieces with higher price sensitivity. Maybe mugs with reindeer are more popular than mugs with mistletoe — so they attract more customers at an even lower price.
Use these testing insights to run new experiments that will determine the ideal price for every item.
Your testing results might surprise you. Customers might respond better to price changes on items with lower base prices. Or, products around $50 could generate just as many sales at $80.
There’s no way to know until you test.
Plus, you can run new experiments to validate your results at any time.
Use a specialized tool for impactful testing
Your price tests are only as powerful as your testing solution.
Resorting to platforms with subpar testing capabilities may require complex workarounds that distort your results, such as duplicating items for each experiment or making backend Shopify adjustments.
But fear not — you can avoid this extra work and guarantee reliable insights with a sophisticated testing tool, like Intelligems.
As the only profit optimization platform built for Shopify, Intelligems empowers you to activate every profit lever in your roadmap. Thanks to its advanced testing methods, you can run seamless experiments and collect accurate data without any dev support.
Plus, you can target customer segments based on factors like device type, country, and currency for granular, intuitive insights.
Intelligems tracks key profit-driven metrics, such as average order profit and profit per visitor, to give you both a high-level overview of your results and a deep dive into the performance of each segment.
Continuously validate your data for optimal margins
The pricing that works today may not work tomorrow — factors like business growth, seasonality, competition, and customer preferences can all affect profitability.
Fortunately, routine A/B testing keeps you ahead of these changes.
Regular experimentation surfaces early shifts in price sensitivity to proactively maintain healthy margins. And for some product categories, this practice is mission-critical to stay competitive:
Low-priced and subscription items (like your monthly wine club), where high LTV comes from repeat future sales
Substitutes (such as Pepsi vs. Cola), where changes to lower-end products often steer customers toward higher-value offerings
Complements (like bagels and cream cheese), where price changes in one item or broader market dynamics would likely impact the other
Use case: Price testing your bestsellers for higher profitability
To maximize bottom-line impact, Original Grain, turned their back on conversion rates — the vanity metric that once led their strategy. Instead, the premium watch brand doubled down on price testing — starting with their 10 best-selling SKUs.
Using Intelligems to effectively set up price experiments and track data, Original Grain discovered customers were willing to pay significantly more than the items’ current prices in more than one instance.
After a few strategic adjustments, the brand increased profit per session on top SKUs by 40%.
🛒 Capitalize on your AOV Builders
Tiered discounts, free shipping, and bundles incentivize larger orders for top-line growth. But for strong margins, you need to look beyond the initial revenue lift. Here’s why:
AOV builders encourage more sales — but risk your profit and conversions
Imagine you offered a volume discount of 10% for orders above $100 and 20% for carts at $150 or more. When evaluating the outcomes, you might notice:
AOV and revenue rise
Ambiguous effect on conversions
Profit per order decreases
On one hand, your offer will likely attract price-sensitive shoppers who wouldn’t have purchased otherwise. But on the other, you risk reducing margins on purchases that would have happened regardless of the price cut.
Even more, your discount might discourage shoppers who need time to decide whether they want to buy more items, delaying conversions and decreasing the bottom line.
At the end of the day, your overall profitability hinges on whether your discount attracts enough customers ready to buy. If the increase in total sales makes up for the decreased price on each sale, then profits will be higher — but if it doesn’t, you’ll be in the red.
A/B testing will pinpoint the lowest threshold for your AOV builders so you can increase conversions, revenue, and profits without giving away more than you can afford. Here’s how to effectively run experiments for this lever.
Inform your tests with historical data
Lean on your store’s customer data to design your experiments. This way, you’ll be testing offers that are more likely to engage your shoppers — rather than taking a gamble on shipping and discount thresholds they may not hit.
For instance, if your items cost around $500 and your shipping is only $10, free shipping might not persuade your shoppers to buy more items — but a free gift for purchases above $600 will.
To spark ideas for more targeted offer experiments:
Look at your AOV data and order distribution to determine the free shipping thresholds customers are willing to reach — such as $150 or $125 if a typical order is around $100
Check which of your products are often purchased together or within a short timeframe to create volume or a bundle discount
Create sophisticated, targeted offers with a robust testing solution
For reliable data, you need to provide customers with a consistent shopping experience. This means each of your testing groups should see the right promotion across emails, external advertisements, and landing pages.
With powerful split testing for tiered discounts, personalized experiences, and email platform integrations, Intelligems has everything you need to achieve this goal.
Picture this: You want to reward regular customers with a special promotion, but you’re unsure which one will perform better — a dollar or a percentage discount.
A/B testing will point out the most profitable discount type and threshold. But Intelligems also enables you to create dedicated landing pages for each offer and connects to your email platform so you can include these pages in your customer outreach.
The results? High data accuracy, delighted customers, and profitable outcomes. A win on all fronts.
Use Case: Testing a free shipping threshold against a lower shipping rate to drive profitability
Historically, Dossier used high shipping rates to make their free shipping threshold more appealing. But they suspected more customers would purchase their affordable luxury perfumes if the initial shipping cost was lower.
With Intelligems, Dossier tested these two options to see which would have a greater impact on profitability. Lower shipping rates was the clear winner, increasing profit per visitor by 19% and helping the brand create a successful shopping journey for its customers.
🎁 Optimize your promotions
Promotions supercharge everyday sales and turn high-stakes sales periods — like the winter holiday season — into powerful growth opportunities.
With the right strategy, you can drive profitability all year round without affecting the sales your customers were already planning to make. But to build effective campaigns, you first need to understand how your promos impact each of your key metrics.
Promotions increase traffic — but your bottom line could take a hit
Promotions have a similar effect on your conversions and margins as your AOV builders:
Revenue, traffic, and orders go up
Ambiguous effect on conversions and AOV
Profit per order decreases
While limited-time offers typically encourage customers to purchase, they might also attract visitors who simply want to check out the promo — offsetting the initial spike in conversions.
This same idea applies to AOV.
Some customers only buy items on sale — decreasing AOV — while others purchase a combination of regular and discounted products, increasing AOV.
When all is said and done, your profitability will increase if your promotion is effective. This means that it needs to bring in enough extra orders to compensate for the lower profit on each order. Only then will your bottom line go up.
To maximize the impact of this profit lever, you need to nail every element of the promotional experience, from the discount amount to the advertising copy. These proven strategies will show you how to fine-tune these details for major holiday promotions and everyday discounts.
Actively promote limited-time sitewide offers for periods with high demand
Potential customers expect attractive sales during holiday shopping events like Labor Day and Black Friday / Cyber Monday.
Give them what they want — implement limited-time offers across your entire store. Auto-applied coupons at checkout and sitewide price cuts are great for this goal.
Identify the right deal with A/B testing. Then, promote it all across the board — on-site, via email, and across paid channels like Meta and Google. This will amplify its effect on your margins.
Attract customers by advertising regular promos on free marketing channels
Periods with lower site traffic call for special promotions that create buzz. Here are two tried-and-true ways to engage your customers during the holiday off-season:
Monthly discounts for single-line products
A dedicated sale section on your site
These offers provide price-conscious shoppers with fresh options to purchase and increase AOV for the rest of your customers. Plus, you can update them to clear out excess inventory whenever you need.
A/B test the elements of your marketing campaigns for additional lift
Experimentation pinpoints the best type of promotion and discount amount for your store. But optimizing the messaging and creative that accompanies these offers will enhance future profits.
Perhaps you’re testing to find the lowest promo threshold for your Christmas campaign.
For your next experiment, try including the following:
Advertising channels like Meta and email
Copy across each channel
Images featured with your copy
The winning combination will turn your campaign into an attractive customer experience, further raising your margins.
Use Case: Securing profitable growth with the right abandoned cart offer
Offering high-end, ergonomic office furniture, Branch needed a way to assure hesitant shoppers they’d found their new workplace staple. Specifically, they wanted to test a 5% against a 10% abandoned cart discount to see which met their goal with a strong bottom line.
Using Intelligems Personalizations, Branch created two different digital experiences for each discount to determine the combination that resonated best with their shoppers. The winning offer and on-site experience increased profits on abandoned carts by 70% and conversions by 80%.
Build your business’ ideal roadmap with a true strategic partner
Seeking out dedicated partners like Intelligems’ Advisory Services can offer the tactical advice and hands-on support brands need to create a robust testing roadmap.
With three support models, Intelligems helps brands at every growth stage optimize their profitability for lasting success:
Strategic Support & Advisory: Custom roadmap design and quarterly roadmap updates, pre- and post-test consultations, and insights into advanced pricing and profitability topics
Strategic Advisory & Analytics: All of the above, plus analytical deep dives pre- and post-testing for major experiments, and dedicated support outside of immediate testing
Full-service Operational Execution: All of the above, plus set-up, QA, test execution, and management of the testing process across your team and external partners
And for their annual partners, Intelligems offers bespoke roadmapping sessions with their team of experts to help you create and execute a custom strategy for every lever on your site.
Pull every profit optimization lever on your roadmap with Intelligems
Intelligems is the only profit optimization tool that lets you activate the most impactful levers across your Shopify store.
With its data-driven and intuitive platform, Intelligems empowers you to build and execute your testing roadmap using features like:
A/B test pricing, promotions, shipping rates, and more
Real-time testing results to see what truly drives profit
Tailored on-site experiences to boost your offer impact
Plus, it's easy to get started. In addition to insightful resources built by Intelligems' eComm experts, the platform offers tailored onboarding to seamlessly integrate with your existing systems. Everything you need to confidently carry out your roadmap.
Ready to take action? We’re ready to help.
Book a demo to start optimizing every aspect of your website for maximum profitability with Intelligems.
Price Testing
Content Testing
Campaigns