Top-Up Screen A/B Testing
Company
PayPay
Industry
Fintech
Timeline
2025/2 - 2025/6
Responsibility
End-to-end design process
Prototyping
Overview
Since joining PayPay, I’ve been responsible for the top-up experience, with a focus on understanding user behavior when adding money to PayPay.
In this project, we ran an A/B test to explore which UI pattern best encourages users to top up larger amounts. This not only increases spending but also reduces top-up frequency, helping to lower operational costs. Over a one-month period, we tested two different UI patterns and gained valuable insights to guide further design decisions.
Results & Prototype
0
0
%↑
Top-up amount
0
0
%↓
Cost of top-up
What a the problem?
We found that users tend to top up small amounts (around ¥5,000) multiple times rather than adding a larger amount at once.
Since each top-up comes with a processing fee. With PayPay’s large transaction volume, these fees quickly add up and cost tens of millions of yen each month.
What do we know about the users?
I further reviewed past research to understand the reasons behind this behavior:
Users are hesitant to keep a large balance in PayPay due to security concerns
The current amount input experience creates friction
Many users lack visibility into their monthly PayPay spending, leading them to top up small amounts repeatedly as their balance runs low
How do we going to solve it?
Based on what we know about our users, we have the following assumptions about how to increase the top-up amount.
Show users their spending from last month and make it easy to top up that amount with a single tap.
However, with multiple design patterns to choose from and no clear evidence on which would perform best, we decided to run an A/B test to validate our assumptions and reduce decision risk.
Idea 1
⚠️ But the problem is, it adds an extra step, users have to leave the top-up flow, check their spending, then navigate back.
Next, we needed to make it easier for users to enter that amount.
Idea 2
⚠️ However, it didn’t work well visually, as it made the layout look off and pushed the other quick amount buttons.
Idea 3
⚠️ But it made bottom bar too large and could potentially cover content at the bottom.
And voilà! Our three testing cases.
Testing case ①
Testing case ②
Testing case ③
Just when things seemed to be going smoothly, an issue came up in Testing case ①
⚠️ After second thought: this felt too aggressive—some users may prefer to enter the amount manually, but when they tapped the placeholder, we automatically filled in a value, which could lead to a poor experience over time.
This made the design less intrusive and maintained the input field as the primary focus in the information hierarchy.
While we’re making it easier for users to input the amount, what if they tap it by accident?
If they didn’t mean to top up that amount, they’d have to hit the delete key multiple times, which could lead to frustration.
It’s a small touch, but I believe it also improves the experience for regular users.
Also from the research mentioned above, one key insight was that users avoid keeping large balances in PayPay because
they don’t feel secure.
Digging deeper, I found that PayPay actually offers a compensation program and several other security measures. The problem, however, is that this information is far too hidden from users.
And we said we were ready to test…
yeah, right. We still need to consider the conditions for showing it, and how often.
We tried to find the answer below:
When should we show it? - Timing Matters
If users see the recommendation in the middle of the month, chances are they’ve already topped up a small amount. At that point, they’re less likely to follow our suggested amount.
To validate this, I reached out to a data team to check the average top-up frequency. The data showed that most users top up about every two weeks, meaning they visit the top-up page roughly twice a month.
So instead of displaying the suggested amount throughout the month, I decided to:
Show it at the beginning of the month, when they’re more likely to make a larger top-up
Limit it to bi-weekly display — shown once every two weeks, and then hide it until the next month. This helps prevent fatigue and ensures the suggestion remains relevant.
What About One-Off Large Spendings?
During QA, I noticed a few users had big spikes in spending, for example, investment-related payments or rare purchases. These aren’t regular behaviors and could throw off the recommended amount.
To keep things accurate, we excluded certain categories from the calculation if they’re clearly not part of regular monthly expenses.
Round the Amount Up — for Tech and Business Reasons
Lastly, I looked at the actual numbers. Users’ monthly spending often ends in awkward figures, like 14,563. Recommending that exact amount didn’t feel helpful or practical.
From both a technical and business point of view, I decided to round up to the nearest thousand. So if a user spent 14,563 last month, they’d see a suggested top-up of 15,000. It’s cleaner, easier to act on, and nudges the user slightly toward a higher amount.
One last thing: if we wanted to test all three options, it would take much more time and a larger sample size.
Since our goal was to move fast and iterate, we decided to start with just 2 options for the initial test. This let us gather insights quickly without slowing down the release.
Player ① Demo
I added an extra quick amount button to make it easier for users to input the recommended amount.
To show last month’s spending, I used the input field’s placeholder—since tooltips aren’t ideal for longer text and can impact readability.
Player ② Demo
I used a banner to display the user’s spending from the previous month, along with a text button that lets them input the recommended amount with a single tap.
Player 1 and Player 2 were each rolled out to 10% of users, while the remaining users saw the standard top-up screen as the control group.
🎉 It’s Player ①
(Top-up amount increased )
while Player ② showed no significant change.
Showing users their last month’s spending can encourage them to top up more.
Adding a banner is visually prominent, but ironically, it was often overlooked (confirmed by CTR data )
The clear amount icon button received positive feedback for making it easy to delete input.
Reflection
I learned that having a clear reason behind each design decision really matters. Even if things don’t work out, understanding the thinking behind them helps you grow and avoid repeating mistakes.
I also found that past research and data can be surprisingly helpful, they often support new ideas in ways you wouldn’t expect.
This also reminded us that design is an iterative process. I shared these insights with the design team, and we’re planning to run more A/B tests in the future.
Next: Cardless ATM →