The Losing Test With a Winner Inside: +A$10,235/Month From PDP Social Proof

Most losing A/B tests get archived and forgotten. This one had a winner buried inside it.

An earlier product page test for an Australian whisky subscription brand had bundled four new elements into one variation: a star rating row, a review carousel, tasting notes, and a trust badge. The bundle did not win. But our post-test analysis said one part of it was almost certainly positive and was being dragged down by the rest.

So we reran the test with only the social proof elements. Result: +15.56% conversion rate with 98% probability of being best, worth an estimated +A$10,235/month in revenue.

Why the First Version Lost

Four elements shipped at once, and the post-mortem pointed at three of them:

• Cognitive overload. The additions substantially lengthened the page and the amount of information a visitor had to process. For a subscription with a simple ask ("join and get this bottle"), more information created more reasons to hesitate, not more reassurance.
• Tasting notes intimidated the wrong segment. The visitor most likely to join a whisky club is the whisky-curious shopper, not the connoisseur. "Medjool dates, cranberry brightness, oily mouthfeel" can make that person feel unqualified rather than excited.
• The review carousel got buried. Sitting below the tasting notes, it landed below the fold on mobile. Too few visitors saw it for it to help, while the extra scroll length still cost something.
• The trust badge was generic. "Australian Made, Small Batch" is accurate but does not differentiate a club membership from buying a bottle at a shop. Noise, not signal.

The star rating was the exception. A 4.9 from 300+ reviews is strong evidence, and our read was that it was net positive in isolation but not strong enough to carry three drags. That read was a hypothesis. The follow-up test was how we checked it.

The Follow-Up: Social Proof Only

Where: Product detail page, all visitors
Tool: Intelligems, inside a Convertibles testing engagement
When: May 11 to July 2, 2026 (Australia/Melbourne)
Traffic: 15,615 visitors who placed 795 orders worth A$121,225

What the control looked like

The standard PDP: product imagery, price, subscription offer, and the purchase CTA. Member reviews existed on the page, but far below the buy decision, and there was no rating summary anywhere near the price.

What the variation kept

• A star rating row directly under the price: "4.9 from 300+ reviews." It is clickable and scrolls the visitor down to the review section, so the claim is one tap away from its evidence.
• A "What Our Members Say" review carousel below the CTA: three static member quotes on desktop, a six-quote swipeable carousel on mobile. Moving it above the spot the tasting notes had occupied put it back in view.

Tasting notes and the trust badge stayed out. Nothing else on the page changed.

15,615 Visitors Later

Metric	Control	Social Proof	Change
Visitors	7,831	7,784	-
Orders	370	425	+66 estimated orders/month
Conversion Rate	4.725%	5.460%	+15.56%
Revenue per Visitor	A$7.20	A$8.33	+15.79%
Profit per Visitor	A$2.35	A$2.68	+14.31%
Average Order Value	A$152.32	A$152.62	No change
Estimated Monthly Revenue	-	-	+A$10,235/month

Conversion rate carried the test: +15.56% with 98% probability of being best, and a confidence interval (0% to +32%) that never dips negative. Revenue per visitor tracked it almost one-to-one at +15.79% (93.2% probability to beat control), and profit per visitor followed at +14.31%, an estimated +A$3,024/month.

Average order value moved from A$152.32 to A$152.62. Flat. Every dollar of the lift came from more people completing the purchase, not from bigger baskets. For a subscription business that is the lift you want, because each of those extra orders is a member relationship, not a one-off cart spike.

What the Retest Actually Proved

The bundle's verdict was about the bundle, not its parts

The first test said "these four elements together do not beat the control." Teams routinely read that as "none of this works" and move on. The retest showed the social proof pair produced a double-digit lift the moment the other elements stopped burying it.

The rating row answers a question at the exact moment it is asked

A visitor looking at a A$150+ whisky subscription is silently asking "do other people think this is worth it?" A 4.9 from 300+ reviews sitting under the price answers that without asking them to leave the buy zone. The click-to-scroll behavior matters too: the summary is a claim, and the review section one tap below it is the receipt.

Reviews below the CTA catch the hesitators, if they can see them

The carousel sits right where a visitor lands when they hover past the buy button without pressing it. Member quotes at that spot speak to the specific anxiety of a subscription: not "is this whisky good" but "is being a member of this club good." In the first test the same quotes existed but sat too deep to do this job.

Before You Copy This Onto Your PDP

Two honest caveats.

First, this works because the rating was real and dense: 4.9 across 300+ reviews. A 4.1 from 12 reviews under your price is not social proof, it is doubt placed at the decision point. Earn the number before you promote it.

Second, position did the work here, not novelty. The reviews already existed on the page. The test moved evidence next to the decision instead of leaving it in the basement. If your PDP already shows a rating summary at the price and quotes near the CTA, this exact test is not your next test.

The Case for Re-Mining Your Losers

Most teams only iterate on winners. But a multi-change loser is not a verdict on every element inside it, it is an average. If the post-test analysis can name a plausible drag (here: tasting notes that intimidate the whisky-curious, a buried carousel, a generic badge), the strongest element deserves its own test before the whole idea gets archived.

This retest cost one testing slot and turned "the new PDP section failed" into "+A$10,235/month, rolled out."

Reader Questions on the Social Proof Retest

Was the conversion lift statistically significant?

Yes. Conversion rate rose +15.56% (4.725% to 5.460%) with 98% probability of being best and a confidence interval of 0% to +32%. Revenue per visitor rose +15.79% with 93.2% probability to beat the control.

Why retest elements from a test that lost?

Because the loss was a verdict on a four-element bundle, not on each element. Post-test analysis flagged the star rating as likely net positive and the other elements as probable drags. Isolating the social proof confirmed it: the pair won on its own.

Did the star rating row or the review carousel drive the result?

They shipped as one variation, so this test cannot split credit between them. What it can say is that the two together, with the tasting notes and trust badge removed, produced the full lift. Splitting the two elements would be the next isolation test if the roadmap demands it.

Why did average order value stay flat?

Because social proof changes whether people buy, not how much they buy. AOV moved from A$152.32 to A$152.62, effectively zero, so the entire revenue-per-visitor gain came from the conversion-rate lift.

Does this apply to stores that are not subscription businesses?

The mechanism travels: put the rating summary at the price and real quotes near the CTA, and make the summary link to the full reviews. The effect size will vary with price point and review quality, which is why it should be tested rather than assumed.

Intelligems powered the test; the hypothesis, build, and analysis came from a CONVERTIBLES CRO engagement. If your store runs on subscriptions, our Shopify subscription CRO service covers exactly this kind of PDP work. Every published win lives in the full test archive, and a 30-minute call gets you three specific recommendations for your own product pages.