This is a mixed-methods UX Research project for the Danish NPO startup and application Inklusiv, as a part of the 1st-semester project case in the Master's Program of Human-Centered Informatics at Aalborg University, Copenhagen.
Company: Inklusiv, a Danish NPO startup and application
Project span: 2,5 months
Group members & responsibilities: Team of 5 from the Master’s program Human-Centered Informatics. I was collaboratively involved in all stages of the UX research process, contributing to a multidisciplinary team effort to achieve research objectives and generate meaningful outcomes.
Process overview: Stakeholder Interviews, Literature Review, User Behavior Analysis, Descriptive Statistics & Data Visualization, Semi-Structured User Interviews (9 participants), Qualitative User Testing Sessions (6 sessions), Thematic Analysis, Synthesis and actionable results list
The startup on a mission to provide inclusive shopping experiences
Inklusiv aims to create inclusive shopping experiences for people living with different types of disabilities. Through a booking system in the app, the customer can book an in-store appointment with a preferred store assistant, at a preferred time, from a shop in the application. Before the physical meeting, the customer is given the opportunity to express their unique needs and preferences to the store assistant in the app, creating better circumstances for a good shopping experience tailored to the customer. The main user groups of Inklusiv are people with different types of disabilities, and retail shops across Denmark from various industries.
As Inklusiv is a non-profit organization getting funding from the government for enhancing inclusivity in society, the application must be making a real change. An essential part of this is making sure the shop assistants and users with disabilities are given the right tools through the app to achieve inclusive shopping experiences.
Booking system: Screen 1: Choosing Time, Screen 2: Describing needs before booking
Defining the problem & research focus
Stakeholder interviews revealed the shop assistants' insecurities and lack of knowledge in interacting with individuals with different disabilities
During the stakeholder interviews, we discovered the problem that during the onboarding process of Inklusiv for shop assistants, numerous users were expressing insecurities and asking questions about how to behave to meet the different needs of customers with disabilities that they would meet through the app. In response, Inklusiv had started creating a part called “Inklusiv Academy” within the app, to provide learning material on different disability types. However, this initiative was not based on any user research or testing, and despite its existence for some time, its effectiveness and impact remained unclear and there was uncertainty about how to continue the development.
Inklusiv Academy learning material
The literature review confirmed and added context to the problem
With the problem in mind, we conducted a literature review on interaction challenges between shop assistants and people with disabilities, which confirmed the issue of insecurities, inadequate training and knowledge among shop assistants. Most importantly, it emphasized that it was one of the main challenges for customers with disabilities in achieving a satisfactory shopping experience. This dual perspective ensured a well-informed approach to address the identified problem further.
Problem statement and research questions:
“How can we bridge the knowledge gap - between store assistants and customers with disabilities, through the Inklusiv app, to enhance the physical shopping experience for both target groups involved?”
Research questions store assistants:
1. What do the shop assistants need to know to provide a good shopping experience and feel comfortable with using Inklusiv?
2. How is the Inklusiv Academy currently working for the shop assistants in educating them about users with disabilities & how can it be improved to ensure convenience for the staff?
Research questions customers with disabilities:
3. What do the users with disabilities want store assistants to know about them before the physical meeting in-store?
4. Is the current way in which users with disabilities provide information about their needs and make a booking through the app convenient? - clarification: Besides Inklusiv Academy, customers expressing needs during the booking process is crucial for shop assistants' understanding & knowledge. Investigating this function, along with confirming bookings, was essential to answer the problem statement.
The research process
1. Quantitative user behavior analysis revealed shop assistants' limited engagement with learning material and a high customer drop-off in the booking process
To understand current user behavior in the Inklusiv Academy and booking process, we analyzed Mixpanel's user behavior log data. This quantitative approach unveiled patterns crucial for identifying potential problem areas for both customers and shop assistants.
Store assistants
For shop assistants, our focus was on Inklusiv Academy usage—whether they even used it, the timing of use, session duration, and page visits. Despite finding out about limited tracking on this part of the app, valuable information on average time per session and page views per session was extracted from the available data.
This showed us that:
● The videos in Inklusiv Academy had low engagement, as the videos are 5 minutes, and the average session duration of 3 minutes for 3 pages indicated users were not watching the full learning material
Customers with disabilities
For this user group, we wanted to find out about the user behavior for making a booking, by digging into the user flow, time on step, conversion rate, and when they dropped off. We also wanted to look into the steps of the booking where the users were providing information about personal needs. To achieve this we conducted a funnel analysis.
The funnel analysis visualization showed us that:
● There is a drop-off in sessions during the booking process, with only 10% of the started booking sessions resulting in confirmed bookings. This indicates potential issues causing users to abandon the process.
● The biggest drop in the booking flow (started from step 2 in the funnel diagram) happened on step 5, (1 of 2 steps of describing personal needs for the visit) where only 48% moved forward.
This was important to look into further qualitatively, to find out the why behind the what, regarding the booking flow problems and low user engagement in the Inklusiv Academy learning material.
2. Descriptive Statistics on Booking Data: Revealed Potential Challenges when customers are describing their needs in the Booking Process & Provided an Understanding of the Customer User Group Demographic
Through applying descriptive statistics on customer booking data in R, we were able to get information about number of bookings, disability types of the users that had made a booking, mean & median number of characters in the needs description steps, as well as connected to disability type. Through understanding the disability types it could guide the learning material needed in the Inklusiv academy, and the number of characters on the needs' pages could indicate potential struggles, connected to disability type to reveal potential accessibility issues.
The data showed us that:
● Many users had more than one disability type, which was an important aspect to keep in mind when creating learning material for the shop assistants in Inklusiv Academy.
● The number of characters in describing personal needs was quite low for all users with disabilities, indicating potential problem areas in the UX, alternatively that they did not feel the need to write that much. This was important to look into further, as the store assistant getting enough information on the needs is important for educating them and creating inclusive shopping experiences.
3. Semi-structured interviews and qualitative user testing to get rich descriptions and new insights through in-depth explorations with shop assistants and customers
We conducted sessions with 9 respondents, involving semi-structured interviews, user testing, and post-testing interviews. Purposive sampling targeted shop assistants and people with disabilities in Denmark—both existing and new Inklusiv users. The outcome: 3 shop assistant interviews and 6 with customers with disabilities. From these, 3 shop assistants and 3 customers participated in user testing.
The interviews aimed at getting detailed descriptions of various problem-related themes, exploring beyond application interaction. User testing provided insights into app interaction, complementing quantitative data by explaining the 'why' behind patterns. This approach uncovered new pain points and recommendations not attainable through a purely quantitative approach. User testing involved presenting participants with a brief scenario and task, instructing them to use the 'think aloud' method.
Shop assistants interview guide themes & user testing tasks
Interview guide Themes: Experiences with customers with disabilities, challenges, current knowledge, lack of knowledge, learning about disabilities, use of Inklusiv Academy
User testing task & Scenario: “You have just received a booking for tomorrow. The user hasn't written much about themselves, just filled in that she has a cognitive disability. You want to prepare for the meeting through Inklusiv Academy. Please pick one of the inklusiv lessons, and watch the full video. Please think out loud when you are navigating in the inclusive academy.”
Post-test questions themes: Based on CTML theory: perceptions on format, & content, its complexity & amount, usefulness, distractions & understanding of purpose. SDT theory: autonomy/control, relatedness, competence.
Customer interview guide themes and & user testing tasks
Interview guide Themes: Disability, shopping behavior, Inklusiv usage, challenges in interaction with staff, lack of knowledge from store assistants about them
User testing task & Scenario: “Navigate to a store of your liking and book a meeting for a future shopping visit - You have to complete all the booking steps and confirm the booking. Please think out loud when you are navigating in the app”
Post-test questions themes: Perceptions and relevance of questions asked in the booking process regarding needs, convenience, interaction experience, perceptions on questions asked about needs in booking flow, recommendations, pain points
4. Thematic Analysis: Unraveling Patterns, Themes, and Insights"
To analyze all of the qualitative data from the interviews and user tests and extract some valid and reliable insights, we conducted two sets of comprehensive thematic analyses, one for each user group data. After transcribing the data, it followed the following process.
First round of coding
The data was coded with a mix of deductive codes based on learning theories and the RQ, and inductive emerging codes (1). They were then organized in a table for each code and participant (2).
Mindmap forming themes
The codes were clustered together, forming themes and subthemes, that were named.
White post its for inductive codes and yellow for deductive codes
Second round of coding
To answer the research questions in a structured way, and to derive insights for each question, the themes were then connected to each research question. After this, a new set of open, inductive coding was done on the data in the themes connected to each research question.
Finding new patterns and themes, leading to insights
An analysis for each RQ was done on the new open codes, by grouping the codes to find new patterns and themes. After naming each of the new groups, the data inside was transformed into descriptions of main insights. The figure is an example of the analysis done for RQ 2.
Grouping codes into patterns and themes and naming them
Converting each named pattern/theme to descriptions of main findings
Putting it all together: Synthesizing insights to actionable findings
The qualitative insights from the thematic analysis were combined with quantitative data to support and complement each other, forming actionable findings. Organized into 'Findings for Inklusiv Academy' and 'Findings for Inklusiv Booking Process,' each finding is followed with an actionable recommendation. This synthesis creates an easily understandable and valuable document for the organization, addressing users’ needs and pain points together with actionable design recommendations to further enhance the app, and move towards bridging the identified knowledge gap.
Findings for Inklusiv Academy learning material
● Store assistants express a need for knowledge across various disabilities, with specific focus areas on communication strategies for hearing impairments and social guidance for psychological/cognitive disabilities.
This conveniently aligns with the booking data that shows that the majority of users securing meetings through the app have sensory or psychological disabilities. (RQ1).
Recommendations:
- Implement the knowledge needs for different disability types in the learning material (from a more detailed report). Prioritize learning materials for sensory or psychological disabilities first.
- Create additional categories for disability types to cater to the diverse inquiries of shop assistants.
● Some users prefer minimal or no communication with shop assistants during the physical shopping experience
"I would like to be shopping anonymously in the shop, without talking much, but still get the help I need. Like, if the shop assistant knows before the meeting what I am looking for"
Recommendations:
- Highlight these varying communication preferences within the same disability in the learning material, focusing particularly on neurodivergent disabilities. Additional recommendations can be found in the "Insights for Inklusiv Booking Process for Customers" section.
● 20 % of users that had made a booking in the app had more than one disability type
Recommendations:
- Consider adding material in Inklusiv Academy emphasizing this, and that the customers thus can't be placed into a category. This is emphasizing the importance of learning about the individual before the meeting.
Findings for Inklusiv Academy format and content
● There is a need for shorter, more concise, and “straight to the point” content to support the store assistants learning in their busy work environments.
They give an example of only showing the user being interviewed in the learning material, and to display the questions as text on the screen. This is supported by quantitative data that show that the average session time for Inklusiv Academy is 3 minutes, while the videos are 5 minutes, and can provide a qualitative explanation.
“If I just think about the training we have for our salespeople, I know that it has to be very concise. And just short information. Because they have other things to focus on.”
Recommendations:
- Create shorter and more straight-to-the-point videos (Around 2 minutes), and remove the distractions mentioned. Work on content engagement content. Test again
● Inclusive Academy's learning material is critiqued for being too individual-focused, hindering a comprehensive understanding of disabilities.
However, they emphasized the importance of being provided individual-focused information about the needs of a customer through the app before a meeting, to create a good interaction with the customer.
“It becomes a bit difficult when you only have the perspective of one person. It's very different for different people."
Recommendations:
- Incorporate multiple perspectives in the learning material to provide a broader understanding. Extract common points from larger studies for more comprehensive coverage.
- As stated by the shop assistants, it is important to learn about the customer’s unique needs that they are meeting, so working on optimizing the customer's experience when providing needs information is essential: See points under “Insights for Inklusiv booking process for customers”
● There is a need for more categories in Inklusiv Academy representing a wider range of disabilities, and subcategories within disabilities to emphasize different needs within disability
However, they emphasized the importance of being provided individual-focused information about the needs of a customer through the app before a meeting, to create a good interaction with the customer.
“Sine can, like, hear, I think, something, but like not fully. So if there was, like, a category for that and another category for like, totally deaf people. And then I know autism is very, like, different from people to people.”
Recommendations:
- Add additional disability categories, with subcategories that capture diverse needs within each disability. For instance, different categories for varying levels of sensory disabilities, showing the unique requirements of individuals within these groups. Adding more categories requires thoughtful categorization.
Findings for Inklusiv booking process for customers
● Funnel analysis shows a high drop off, with only 10% of users finishing booking of those who starting the booking.
The biggest drop in the booking flow happened on step 5, where only 48% moved forward. This was the first of 2 steps in describing the needs for the visit
Recommendations:
- Conduct an accessibility evaluation on the whole booking flow, so that all customers can move through the flow with ease.
- The qualitative insights below can help explain this, and the recommendations can provide solutions.
● Adding descriptions about needs can feel repetitive for recurring bookings
The biggest drop in the booking flow happened on step 5, where only 48% moved forward. This was the first of 2 steps in describing the needs for the visit
"If I would use the app more often I think the process is too long. Or like I think in this field what I need help with I would probably always write the same thing over again.”
Recommendations:
- Add an option for users to write their needs in their profile (with the option to be displayed anonymously) so that they can reuse this description in the booking process through autofill.
● Unclear questions: Some users expressed that they are not sure about what specifically to include when expressing their needs.
The biggest drop in the booking flow happened on step 5, where only 48% moved forward. This was the first of 2 steps in describing the needs for the visit
“Sine can, like, hear, I think, something, but like not fully. So if there was, like, a category for that and another category for like, totally deaf people. And then I know autism is very, like, different from people to people.”
Recommendations:
- Specify what the users can write about (examples), and create predefined alternatives. Emphasize the aim of asking about this and the value of the shopping experience. Make it possible to add needs after booking as well.
● Some users expressed a preference for minimal or no communication with shop assistants during the physical shopping experience
Recommendations:
- Add options in the booking process to choose the level of communication with shop assistants in the physical interaction, for example with checkboxes, where one option is displayed as: “little to no interaction”. This would also meet the shop assistants' need regarding knowing how social and straightforward to be.
Key learnings and way forward
The outcomes of this project can serve as a valuable resource for future app enhancements. A strategic next step would involve designing a prototype with the recommended changes and testing it on users for additional feedback. User behavior patterns should be followed up on through Mixpanel to assess potential positive changes, such as improvements in metrics like time spent on videos. Additionally, an accessibility evaluation should be conducted and tested together with the customers with disabilities on the whole app to ensure a smooth and truly inclusive digital experience for all users. Through creating a possibility to add feedback after the physical visit for customers, we could also see the quality of their upcoming customer experiences.
Implementing the design recommendations for the learning material has the potential to better prepare shop assistants to provide the right service to the customers with different disabilities and meeting their needs. By incorporating recommendations for the UX of the booking process for customers, they may feel more at ease completing the booking and providing a description of their needs. This, in turn, educates shop assistants, contributing to a more inclusive and satisfactory shopping experience for customers.
Reflecting on the journey, a key learning was the significance of initiating participant recruitment from the project's start. Recognizing this, I would adopt an early recruitment strategy in future projects to overcome time constraints and secure a more robust participant pool for interviews and user tests.
