Shipping Protection Toggle: +24% Conversion With the Fee Off by Default
[ SUMMARIZE WITH AI ]
[ FREE CRO TEARDOWN ]
Find the 3 biggest revenue leaks on your store.
Every day a conversion leak goes unfixed, you're paying for traffic that doesn't buy. Get a 5-minute Loom through your PDP, cart, and checkout, with mockups of the fixes. No pitch.
Get My TeardownShipping protection should be a toggle, not a fork in the road. And when we tested which way that toggle should start, the off position won.
A consumer electronics accessories brand had a shipping protection app sitting at the bottom of its cart drawer. The app's stock layout ended the drawer with two big buttons: one to check out with protection, one to check out without it. Every shopper had to pick a side before paying.
We replaced the fork with one shipping protection toggle and a single checkout button whose total updates live. Then we tested two versions: toggle on by default, and toggle off by default.
19 days. Control plus two variations, traffic split evenly three ways.
With shipping protection off by default, conversion rate rose 23.7% at 97% probability to beat control. Revenue per visitor rose 16.4%, at 82%.
Two Buttons, One Bad Choice
Most shipping protection apps install the same way. A row with the fee, a line of microcopy, and a primary button that already includes protection. Underneath, a smaller "continue without" link or button.
From the app's side, that layout makes sense. It sells protection.
From the shopper's side, it reads like a trick. The big button total is higher than the cart subtotal they just saw. To pay the price they expected, they have to find and click the smaller option. That is the moment people start wondering what else was added.
Our hypothesis: an opt-in toggle with one honest total would reduce that "am I being upsold?" feeling, cut drawer abandonment, and lift conversion rate and revenue per visitor.
The open question was the default. On keeps more protection fees. Off feels cleaner. We did not want to guess, so both went into the test.
Control Plus Two Variations: Same Toggle, Different Default
Component: Cart drawer footer, sitewide
Platform: Intelligems
Test type: Control plus two variations
Duration: 19 days
Traffic: Split evenly three ways
Devices: Desktop and mobile
| Version | Drawer footer | Protection default |
|---|---|---|
| Control | App's stock widget: fee row, microcopy, "checkout with protection" button plus a "continue without" button | Shopper must choose |
| Variation 1 | One row with the fee, an info icon and a toggle. One checkout button showing the live total | On |
| Variation 2 | Identical to Variation 1 | Off |
The two variations differ by exactly one thing: where the toggle starts. Our team built both as one component with a single setting for the default, so nothing else could drift between them.
Two scope notes. Only the cart drawer changed; the full cart page kept the app's stock widget for everyone. And we never touched the app's pricing logic. Our checkout button calls the same functions the app's own buttons call, and reads the fee straight from the app, so the shipping protection price shoppers saw was always the app's price.
Whole-Store Numbers: Default Off Took the Lead
| Metric | Control | Variation 1 (on) | Variation 2 (off) |
|---|---|---|---|
| Conversion rate | 1.53% | +6.8% (71%) | 1.90% (+23.7%, 97%) |
| Revenue per visitor | baseline | +8.2% (69%) | +16.4% (82%) |
| Average order value | $148.42 | not reported | $139.58 (-6.0%) |
Percentages in brackets are probability to beat control.
Conversion rate on Variation 2 is the one number here that cleared 95%. Revenue per visitor points the same way but sits at 82%, which is directional, not proven. The order counts behind it are still small.
Average order value dipped 6% on Variation 2. Our read: part of that is the shipping protection fee itself, which fewer Variation 2 orders carried. We cannot split the fee out of this data, so treat that as a likely explanation, not a measured one.
Most of the conversion lift came from phones. The two-button layout takes up most of a phone screen, so removing it matters more there.
Shoppers Who Opened the Cart Tell a Messier Story
The whole-store view counts every visitor, including the ones who never opened the cart drawer. We also tracked the shoppers who actually saw the drawer, because that is where the change lived.
In this lens the leader flips. Variation 1 edges ahead of Variation 2, on a few hundred drawer viewers per version.
We are showing you both views because hiding the second one would make this case study cleaner and less true. Here is what we take from it:
- Both variations beat control in both lenses. The toggle plus one total is the real win. That part is solid.
- On versus off is not settled. A few hundred drawer viewers per version cannot separate a gap that small between two variations. Neither lens proves one default beats the other.
- Defaults stick. Very few shoppers flipped the toggle at all. Whatever position you ship is the position almost everyone pays.
So the brand shipped Variation 2. It had the only result above 95% (whole-store conversion), it led on revenue per visitor across the whole store, and it does not pre-select a paid add-on. When the data cannot split two options, pick the one that does not charge people by default.
The Shipping Bug That Only Exists When You Pick "Off"
Moving a test winner to live code usually means cleanup. This one surfaced a real risk.
The shipping protection app treats a missing setting as "protection on". A fresh cart has no setting yet, so if the write that turns protection off stalls or fails on the way to checkout, the shopper gets charged for protection behind a toggle that says off.
During the test that path was rare. Once off became the default for all traffic, it became the main path. Our QA review caught it before launch. The fix: confirm the off setting actually landed before sending anyone to checkout, with a direct fallback write and a hard time limit so the button can never hang.
The same review added one more guard. If the toggle's script fails for any reason, the drawer falls back to the app's stock buttons instead of showing a checkout button that does nothing.
If you copy this test, copy these two checks too. A default-off toggle that silently charges people is worse than the two-button layout you started with.
Shipping Protection Default Questions
Should shipping protection be on or off by default?
Off, unless your own data says otherwise. In this test the off default won on whole-store conversion at 97% probability to beat control. Among shoppers who opened the cart drawer, the on default edged ahead, but the gap was too small to call. With no clear winner between the two, off is the default that does not pre-select a paid add-on.
Does a pre-checked shipping protection fee hurt conversion?
In this test, the bigger problem was the two-button layout, not the default. Both toggle versions beat the stock layout. The on-by-default version still lifted whole-store conversion 6.8%, but only the off-by-default version reached 97% confidence.
Why did average order value drop when protection was off by default?
It fell 6.0%. The most likely reason is that fewer orders included the protection fee, since almost nobody changed the default. We could not separate the fee from the rest of the order in this data, so that is an explanation, not a measurement.
Is 82% probability to beat control enough to ship a revenue per visitor result?
Not on its own. Here it came with a conversion lift at 97%, both lenses agreed the new layout beat control, and the change was cheap to keep. That combination made shipping the better call than running longer. On a change that is costly to reverse, wait for 95%.
Do I need custom code to test a shipping protection toggle on Shopify?
We built a custom drawer component that drives the app through its own functions, so fees and totals always came from the app. Some shipping protection apps offer a toggle mode in their settings, but it renders their own design, not the one we tested. Whichever route you take, check what the app does when the setting is missing before you ship a default-off version.
How much traffic does a shipping protection test need?
More than you think, because the change only affects shoppers who open the cart. In this test only a small share of visitors were tracked opening the drawer. Plan on at least a few hundred drawer viewers per variation, and expect the drawer-level read to stay noisy even then.