Top-Up Screen A/B Testing
Company
PayPay
Industry
Fintech
Timeline
2025/2 - 2025/6
Responsibility
End-to-end design process
Prototyping
Overview
Each top-up at PayPay carries a processing fee. At millions of transactions a month, small inefficiencies in user behavior add up to tens of millions of yen in operational costs.
The pattern I noticed: users were topping up small amounts repeatedly, around ¥5,000 at a time, instead of adding a larger balance upfront. This pointed to two product gaps: no spending visibility at the moment of top-up, and a friction-heavy input experience.
I ran a one-month A/B test to find out if surfacing last month's spending at the right moment could shift that behavior.
0
0
%↑
Top-up amount
0
0
%↓
Cost of top-up
What is the problem?
We found that users tend to top up small amounts (around ¥5,000) multiple times rather than adding a larger amount at once.
Since each top-up comes with a processing fee. With PayPay’s large transaction volume, these fees quickly add up and cost tens of millions of yen each month.
What do we know about the users?
To better understand why users top up in small amounts, I reviewed past research and user data, and identified 3 key patterns:
Users are hesitant to keep a large balance in PayPay due to security concerns: this came up repeatedly in user interviews, suggesting that the barrier isn't just habitual, it's emotional.
The current amount input experience creates friction: usage data showed drop-offs at the input step, indicating that the UI itself was discouraging larger entries.
Many users lack visibility into their monthly PayPay spending: without a clear reference point, users default to topping up small amounts reactively as their balance runs low, rather than planning ahead.
How do we going to solve it?
Based on these findings, we formed a core hypothesis:
if users could see their last month's spending at the moment of top-up, they'd have a concrete reference point and be more likely to top up a larger, more intentional amount.
We explored 3 variants test this, each with a different approach to surfacing the information and reducing input friction.
Ideation
Variant A
But the problem is, it adds an extra step, users have to leave the top-up flow, check their spending, then navigate back.
Next, we needed to make it easier for users to enter that amount.
Variant B
However, it didn’t work well visually, as it made the layout look off and pushed the other quick amount buttons.
Variant C
But it made bottom bar too large and could potentially cover content at the bottom.
This gave us three testing cases to move forward with.
Just when things seemed to be going smoothly, an issue came up in variant A
After second thought: this felt too aggressive — some users may prefer to enter the amount manually, but when they tapped the placeholder, we automatically filled in a value, which could lead to a poor experience over time.
This made the design less intrusive and maintained the input field as the primary focus in the information hierarchy.
While we’re making it easier for users to input the amount, what if they tap it by accident?
If they didn’t mean to top up that amount, they’d have to hit the delete key multiple times, which could lead to frustration.
This reduces friction for users who enter an amount manually and need to correct it quickly.
They don’t feel secure.
Also from the research mentioned above, one key insight was that users avoid keeping large balances in PayPay because
We've now ideated 3 variants, time to dive into the details and edge cases.
When should we show it?
If users see the recommendation in the middle of the month, chances are they’ve already topped up a small amount. At that point, they’re less likely to follow our suggested amount.
To validate this, I reached out to a data team to check the average top-up frequency. The data showed that most users top up about every two weeks, meaning they visit the top-up page roughly twice a month.
So instead of displaying the suggested amount throughout the month, I decided to:
Show it at the beginning of the month, when they’re more likely to make a larger top-up
Limit it to bi-weekly display — shown once every two weeks, and then hide it until the next month. This helps prevent fatigue and ensures the suggestion remains relevant.
What About One-Off Large Spendings?
During QA, I noticed a few users had big spikes in spending, for example, investment-related payments or rare purchases. These aren’t regular behaviors and could throw off the recommended amount.
To keep things accurate, we excluded certain categories from the calculation if they’re clearly not part of regular monthly expenses.
Round the Amount Up — for Tech and Business Reasons
Lastly, I looked at the actual numbers. Users’ monthly spending often ends in awkward figures, like 14,563. Recommending that exact amount didn’t feel helpful or practical.
From both a technical and business point of view, I decided to round up to the nearest thousand. So if a user spent 14,563 last month, they’d see a suggested top-up of 15,000. It’s cleaner, easier to act on, and nudges the user slightly toward a higher amount.
Tradeoff
We initially explored 3 variants. Testing all three simultaneously would have required a significantly larger sample size and extended the timeline, which didn't align with our goal of iterating quickly. So we moved forward with Variant A and Variant B.
These two represented the most distinct interaction patterns. The contrast between them would give us the clearest signal about what actually drives behavior change.
Variant A Demo
Variant B Demo
Variant A and Variant B were each rolled out to 10% of users, while the remaining users saw the standard top-up screen as the control group.
Results
Variant A won.
Top-up amount per user: +7%
Top-up frequency: -22% (from 4.5 to 3.5 times/month), reducing per-transaction processing costs at scale
Variant B showed no significant change
Reflection
Our core hypothesis was confirmed: showing users their last month's spending at the moment of top-up does shift behavior. But the result came with an important nuance. Variant A, which surfaced the information through the input field, drove a 7% increase in top-up amount. Variant B, the banner with an explicit CTA, showed no significant change.
The same information, presented differently, produced completely different outcomes. Users entering the top-up flow already have a number in mind. A banner at that point feels like an interruption. An input field suggestion feels like a natural reference.
This points to our next hypothesis: if we surface spending context before users enter the top-up flow, when they haven't yet anchored to a number, the nudge may be more effective. We're planning to test this as the next intervention point.
Next
Cardless ATM
Expanded ATM features to eliminate the need for debit cards, resulting in a 12% increase in service adoption.
