What they needed
As part of a 3-stage testing process, Wayve has a team of highly trained, highly-skilled drivers - AVSO’s (Autonomous Vehicle Safety Operators) - who conduct on-road testing of the AI software.
These AVSOs monitor how the self-driving technology performs on public roads, and are on hand to step in to correct it if needed (by switching the car back to manual driving or overriding the self-driving features at any time).
On-road testing is essential to the development of self-driving technology, and Wayve runs 100s of on-road tests every week.
Our job was to identify how to improve the AVSO user experience, and the reliability and accuracy of the data that the on-road testing provides their in-house research and software engineering teams.
“On-road testing is essential to developing and validating our self-driving technology. It provides real-world data that can only be obtained by driving on public roads. We started testing on public roads in London and other UK cities in 2018. Our daily on-road trials take place with an expert safety operator behind the wheel at all times.” – https://wayve.ai/safety/how-we-test/
(Image source: https://wayve.ai/. ©Wayve Technologies Ltd.)
What we did for them
Through a stakeholder interview, ethnographic research, and user interview that enabled us to understand the issues AVSO’s experienced with the test software, and a UX workshop, we were able to uncover a number of areas that could be improved.
Ethnographic research
Fruto’s UX researchers spent a morning with a Wayve AVSO to get a better understanding of how the AVSOs interacted with the software, and what it was like to monitor an autonomous vehicle.
Joining a test drive around London allowed us to observe the AVSO's overall experience and interaction with the interface that was potentially influencing the quality of the data that the on-road testing provided the research team.
Conducting the interview in this way - with the user in their natural physical environment, completing a real test drive - meant that the AVSO could both show and tell the details of their experience, including occasional workarounds to certain challenges they faced.
This provided our researchers with more detailed, holistic research insight into the end-to-end user experience.
Stakeholder workshop
Having completed the contextual user interview, we then ran a half-day user journey mapping workshop with stakeholders.
We started the session by delivering a short introduction to UX. This ensured that workshop participants understood the reasons behind the UX research, and provided them with a better understanding and awareness of the importance of user experience design to take away and apply to their everyday work activities and decision-making.
Bringing people from across different teams of the company together to holistically look at the AVSO experience and map out the user journey (identifying what goes well and what could be improved), really helped us further understand both the problems and opportunities with the testing experience and data it produced, and the potential solutions.
Journey mapping provided a collaborative platform through which we could facilitate Wayve's team to think about how they conduct their on-road experiments.
Including members of Wayve’s research and engineering teams in the workshop enabled us to fully understand the importance and accuracy of the data that the on-road tests provided
Having the same user (AVSO) from the interview partake in the workshop allowed us to ask questions directly relating to the observed experience and refer back to incidents in the specific context.
Workshop attendees were all highly engaged in the activity, which resulted in creating over 100 Post-It notes relating to the daily journey of the expert drivers from multiple different perspectives.
“I found it very useful to map the journey. I felt like I should know most of what we uncovered, but actually, it shed new light on the UX. I now have more context and the ability to articulate the phases of the experience.”
– Workshop attendee
Digitalising this journey map meant we had a permanent record to refer back to when analysing the data and pulling together a recommendations report.
Recommendations Report
Analysing the data collected in the user research session, we identified some potential improvements in the user journey - some of which were quick wins that would be relatively fast to implement, whereas others would require more intensive design and development work.
For example: Supporting smoother experiment feedback experience
We analysed the process of an AVSO capturing post-experiment feedback for researchers. Existing approaches provide good data but can be time-consuming and add cognitive load and context switching for the people entering complex information:
The quick-win solution: Provide no-typing input to provide feedback easier.
The longer-term solution: Co-creation workshop to redesign the feedback loop.
The results of our findings were presented back to Wayve's team in the form of a recommendations report.
Results
Some improvements to the assignment flow have already been put into practice, with plans now being made around tackling the challenges that require a more strategic design solution.
It was a fantastic workshop with the Fruto team, we came out with some real practical recommendations which immediately improved our workflows. Additionally, our wider team gained an understanding of the complexities and frustration of the user process and gained the tools to explore how we can improve it together.
Ben Foxall, Team Lead (Robot Web) at Wayve
As designers, it’s really exciting to be involved with projects that involve working with emerging technologies such as AI and machine learning. And Wayve is a great example of how investing in UX expertise leads to better, more user-centred digital experiences.
If you’ve got a similar project you’re interested in us working on, please get in touch.
