De-Risking Push Authentication: Realistic Testing with 40+ AIB Customers
Real-world testing with personal banking customers to validate a new authentication flow and de-risk business decisions.
Overview
The Customer Experience Strategy (CES) team at AIB wanted to streamline identity verification for customers calling the contact centre. Their idea: let customers authenticate themselves using a push notification from the AIB mobile app, allowing them to skip security questions and get faster help from advisors.
But there were key risks to test:
-
Could customers manage switching between a phone call and the mobile app?
-
Would they trust and respond to a push notification during a call?
The design team was brought in to prototype and validate this solution with real users, helping de-risk the approach before rollout.
Role & duration
My role
As part of the UX design and research team, I played a hands-on role in prototyping, planning, and running usability sessions at AIB’s in-house research facility, The Lab.
My contributions included:
-
Designing and iterating interactive prototypes (low- to high-fidelity) to simulate real customer calls and app interactions
-
Collaborating with designers, stakeholders, and lab staff to set up testing logistics and recruit participants
-
Facilitating live usability sessions, observing user behaviour, and probing for deeper insights
-
Synthesising findings and highlighting usability pain points during team debriefs
-
Helping the team align on changes to the IVR script and app flow based on customer feedback
This project gave me valuable experience leading in-person research, collaborating across teams, and using early insights to influence product direction.
Timeline
April – June 2022
The problem
AIB personal banking customers who call the contact centre seeking support with their card (e.g. if they have been a victim of fraud) spend a long time waiting to speak to an agent. Because a large number of calls require manual identity verification by the agent (taking on average 30-60 seconds per call), average call handling times are high and agents are unable to promptly get to each customer’s query. This situation causes frustration for both customers and contact centre support staff and represents an avoidable cost to the business (increased cost to serve).
Design
Happy Path
The CES team came to the design team with a proposed IVR Script, and wanted us to design screens which would support the happy path flow. The happy path flow involves steps such as the user calling the Phone Banking support number, entering the required info on their device’s keypad, receiving the push notification, interacting with it, entering their PAC in the AIB app and returning to the call.
The CES team wanted us to test this experience out with customers in as close to a real life simulation as possible. They wanted to identify if customers would have any difficulties with:
- The interaction of listening to the IVR voice/instructions,
- Taking the phone away from their ear to enter details on the keypad,
- Hearing/seeing the push notification come in, and being able to successfully action it.
We began by mapping the basic flow out in Invision Freehand in low fidelity wireframes in a workshop with stakeholders from the CES team, before taking it to a higher fidelity.
Unhappy Paths
Although we considered a number of unhappy paths, it was not possible to test these, for example:
#1 User does not have their registered mobile device
#2 Customer is outside of data/wifi zone
In these situations, the IVR is designed to put the customer through to an agent for manual identification and verification.
Creating a realistic prototype
Getting creative with tooling
At the beginning of the prototyping phase, we encountered a problem – the tools our design team typically use (Sketch & Invision) did not support text to speech & being able to realistically simulate a push notification.
We began looking at other tools which could support this and found Protopie.
Using Protopie, we were able to simulate a real phone call to the contact centre. The text-to-speech functionality in Protopie enabled us to realistically simulate an Interactive Voice Response (IVR).
Protopie allowed us to program the prototype with pre-determined audio responses based on user’s input on the device keypad.
It also enabled us to realistically simulate a push notification, from the audio ‘ping’ and vibration on the device, to the slide in animation as it came down from the top of the user’s screen.
We built two different prototypes which were slightly different (iOs and Android) in line with how each operating system is designed and works.
iOs prototype:
Android prototype:
Testing
Planning with cross-functional stakeholders
The design team held a workshop with the stakeholders on the CES team to plan the research. A rainbow spreadsheet was created with expected observations, in conjunction with the CES stakeholders.
This would allow us to quickly identify trends visually where user behaviour deviated from what was expected i.e. quickly identify problems & where they are experienced.
We educated the stakeholders in how to use the usability testing observation tool so that they could actively participate on the day of testing.
Test scenario
In collaboration with the stakeholders, we planned our test scenarios and instructions for users, that are as follows:
- You wish to get some help from AIB regarding a problem you are having. You have been a victim of fraud and need to block your card, using the device provided (imagine it is your own), please call AIB. Follow the instructions as directed.
- Your AIB registration number is 3377 8889. Your PAC is 12345.
Follow up questions
- How did you feel going through that process?
- Do you know your REG/PAC off by heart?
- If you didn’t have your REG/PAC to hand during the phone call what would you do? What would you expect to happen?
- What would you do if you didn’t receive the push notification?
Guerilla usability testing with 40+ customers
Together with another UX designer and our CES partners, we set up a day of in-person usability testing at AIB’s Digital Lab in Dundrum Town Centre. Our goal was to observe how real customers would handle the push authentication flow in a realistic setting.
We approached people as they exited the lab. This ensured we were getting current AIB customers, which we confirmed with each participant.
We did not collect any personal/idenitfying details, each participant was given a number on our rainbow spreadsheet which protected their anonymity. Test devices were used (x1 Android and x1 iOs) in which we had pre-installed Protopie app. COVID-19 precautions were undertaken.
Key findings
80% of participants could successfully self-authenticate
The majority of customers (80%) could hear and follow the instructions on the call, clicked on the push notification and successfully enter their PAC in the app to self authenticate.
This helped the team to validate the UX feasibility of the proposed solution.
Challenges & changes
Only 8/40 users knew their registration number off by heart. Participants mentioned that due to a recent app update (whereby registration number is now auto-populated when logging in) they no longer need to know it.
This was a major finding and resulted in a change to the IVR script whereby mobile number is now asked for to authenticate the customer (vs registration number).
Some customers equated push notifications with scam texts, highlighting the need for clearer education and messaging around security.
Impact
Providing clarity and reducing risk
Through realistic prototyping and usability testing with over 40 AIB customers, our team validated that push notification authentication was feasible for most users.
Key outcomes included:
-
Reduced call handling time: By confirming that customers could self-authenticate, the team moved forward with confidence, expecting shorter verification steps and faster support for callers.
-
Improved IVR flow: Testing revealed that only 20% of users knew their registration number, leading to a key change — the IVR script was updated to request a mobile number instead, improving success rates.
-
Increased focus on customer trust: Customer concerns about fraud and scams highlighted the need for clearer education and reassurance when introducing new verification methods.
By surfacing these insights early, our research helped the business de-risk their approach, align cross-functional teams on improvements, and move closer to a more seamless and secure customer support experience.
Reflections & learnings
The power of contextual research & cross-functional collaboration
This project taught me the value of designing realistic testing environments to tackle risky design decisions with confidence. By simulating the full phone call and app-switching flow, we were able to surface issues that wouldn’t have emerged in simpler tests — from technical hurdles to moments of user mistrust.
Key personal learnings:
-
High-volume in-person testing: I learned how to manage fast-paced usability sessions effectively, balancing structure with flexibility as each participant brought unique challenges.
-
Cross-functional collaboration: I saw the power of cross-functional collaboration in action. Bringing the CES stakeholders along the journey from planning testing to actually testing the solution with customers helped us to quickly align around customer pain points and next steps.
-
Reading between the lines: I developed sharper instincts for probing subtle signs of confusion or hesitation, especially when users’ first answers seemed positive but their behaviour told another story.
Overall, this experience deepened my understanding of how early, realistic customer feedback can directly shape product strategy and reduce business risk.