UX Research

The Design Problem

The IT stakeholders started complaining about that 9000 users didn’t use their new internal product that would make the work for Swedens largest government agency easier. Though the product was more a MVP than a fully usable product they insisted that the user should use that program and they thought about blocking the old software even though the user had to use the old software due to limitations in the new one. 
I was tasked with finding out why the 9000+ users didn’t use the product and became UX Researcher for the short project. 

After conducting a natural set of 9 user observations at their working desks it came apparent what was wrong. The product was later terminated for an off the shelf platform saving precious development time.

My strategy

As  a Product Owner and UX designer of the new booking system it got assigned to investigate the problem. I started with UX research and called a contacts at an office and made an appointment to come and run some natural, contextual qualitative observations. 

To prepare I opened up the users digital workplace as to do desktop research and asked these questions:

• How does the user journey look like for the old software and how does the user journey look like for the new software?

• Are their clear pain points that was easy to spot?

• What might be confusing?

• What other software did they use in their work?

• Could they do their job without the need of the new software?

By now I had empathised with the user in the process of Design Thinking as well as Ideating possible user journeys that might be the problem. But it was only a guess as to what might be the problem. 

Since I do not work with guesses or support subjective opinions I went for the Data Driven Design and set of to do contextual natural qualitative observations to find the real cause of the problem since the agency had payed a lot of money developing the user software that would simplify the work.  

Goal: To find out why they are not using the new system for working with customers or clients.

Hypothesis H0: They have a mental model that simply follows the architectural design in the user journey through out the system.

H1: They have a mental model where they open up the new software in combination with the old system. 

Qualitative UX Research - Natural Observations

I new that their would be at least five users in the team with maybe a drop off of five users due to sick leave or parental leave due to sick children or other causes. Five users usually is fine to see a pattern. 

I came in to their office early as to be set up before the users would arrive as to be able to watch them in the team walking in taking of their jackets and starting up their workplace. 

As they came in I greeted them and since they already new the purpose of my visit I told each participant to think out loud started following each one as the sat down starting their computer and started documenting their user journeys. After seven observations the pattern was very obvious.

Semi- Structured Interviews

So I started to ask questions about why they choose the user journey that they did. After nine interviews it was apparent what was going on so and I headed back to the office to analysis the research.

Thematic Analysis

The user journeys of each user told the story about why the result was showing no usage of the new software at all. 

The whole IA - Informational structure or the architecture from the agency internal homepage from where you start all the software that you use in your work was not thought through since they had different product owners and product managers. 

All working in silos not really caring about other programs or how the user would use the software together. The agency had little to no education or experience with UX Design and it was very apparent that it was so.

Conclusion

Hypothesis

H0: They have a mental model that simply follows the architectural design in the user journey through out the system.

Test=Fisher Exact test since N is less than 50.

Users=N=7

Expected=7

Actual=7

Null hypothesis could not be rejected since p-value with Fishers Exact test = 1.

P-value of less than 0,05=5% is required for us to reject H0. 

100% of the users followed the mental model of the architectural design in the user journey and thus never opened up the new software. 

Why?

The reason for the user not using the new software was twofold. 

Firstly

The first reason was that when the user started its workday it wanted to get a handle on how their day would look like as to be able to plan out all the work. In order to do that their user journey started with:

1. Starting computer.

2. Opening internal homepage.

3. Starting up the software that had information about their booked appointments during the day.

4. Opening up the first appointment.

5. Clicking on the first booked appointment.

6. Clicking on the first booked appointment customer.

7. The first customer id was coded with the API to the old software.

I guess you have already figured it out. The whole informational architecture and API connected software bypassed the new software and no one had cared to work cross functionally as to make the complete user journey efficient and useful.

Secondly

On the homepage where you hade all the links to all the internal software. There was a grid with all the programs icons. All shifted to the right of the screen. The grouping was fine and that itself followed Gestalt rules, the law of proximity and so on. But the colors contrast of the icons and level of control of the icons was zero. 

It turns out that even if you new you had to use the new software the color and contrast of the icon was so bad that you had a hard time distinguishing it from the rest of the icons or even find it. It was buried in the second row on the third spot nicely blended together in an ocean of other low contrast icons with zero “skeuomorphism” as to convey what the icon was connected too. 

So even if you wanted to use the new software you would have a very hard time finding the icon and understand that you found the right one since you had to use your memory rather than recall as to identify and click on it. It no affordance so to speak.

Time spent on this problem solving UX Research: 8 hours

Cost: 8 x 800 SEK/hour = 6400 SEK off the budget.