This website uses cookies to better the user experience of its visitors. Where applicable, this website uses a cookie control system, allowing users to allow or disallow the use of cookies on their computer/device on their first visit to the website. This complies with recent legislative requirements for websites to obtain explicit consent from users before leaving behind or reading files such as cookies on a user’s computer/device. To learn more click Cookie Policy.

Privacy preference center

Cookies are small files saved to a user’s computer/device hard drive that track, save, and store information about the user’s interactions and website use. They allow a website, through its server, to provide users with a tailored experience within the site. Users are advised to take necessary steps within their web browser security settings to block all cookies from this website and its external serving vendors if they wish to deny the use and saving of cookies from this website to their computer’s/device’s hard drive. To learn more click Cookie Policy.

Manage consent preferences

These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
Cookies list
Name _rg_session
Provider rubygarage.org
Retention period 2 days
Type First party
Category Necessary
Description The website session cookie is set by the server to maintain the user's session state across different pages of the website. This cookie is essential for functionalities such as login persistence, ensuring a seamless and consistent user experience. The session cookie does not store personal data and is typically deleted when the browser is closed, enhancing privacy and security.
Name m
Provider m.stripe.com
Retention period 1 year 1 month
Type Third party
Category Necessary
Description The m cookie is set by Stripe and is used to help assess the risk associated with attempted transactions on the website. This cookie plays a critical role in fraud detection by identifying and analyzing patterns of behavior to distinguish between legitimate users and potentially fraudulent activity. It enhances the security of online transactions, ensuring that only authorized payments are processed while minimizing the risk of fraud.
Name __cf_bm
Provider .pipedrive.com
Retention period 1 hour
Type Third party
Category Necessary
Description The __cf_bm cookie is set by Cloudflare to support Cloudflare Bot Management. This cookie helps to identify and filter requests from bots, enhancing the security and performance of the website. By distinguishing between legitimate users and automated traffic, it ensures that the site remains protected from malicious bots and potential attacks. This functionality is crucial for maintaining the integrity and reliability of the site's operations.
Name _GRECAPTCHA
Provider .recaptcha.net
Retention period 6 months
Type Third party
Category Necessary
Description The _GRECAPTCHA cookie is set by Google reCAPTCHA to ensure that interactions with the website are from legitimate human users and not automated bots. This cookie helps protect forms, login pages, and other interactive elements from spam and abuse by analyzing user behavior. It is essential for the proper functioning of reCAPTCHA, providing a critical layer of security to maintain the integrity and reliability of the site's interactive features.
Name __cf_bm
Provider .calendly.com
Retention period 30 minutes
Type Third party
Category Necessary
Description The __cf_bm cookie is set by Cloudflare to distinguish between humans and bots. This cookie is beneficial for the website as it helps in making valid reports on the use of the website. By identifying and managing automated traffic, it ensures that analytics and performance metrics accurately reflect human user interactions, thereby enhancing site security and performance.
Name __cfruid
Provider .calendly.com
Retention period During session
Type Third party
Category Necessary
Description The __cfruid cookie is associated with websites using Cloudflare services. This cookie is used to identify trusted web traffic and enhance security. It helps Cloudflare manage and filter legitimate traffic from potentially harmful requests, thereby protecting the website from malicious activities such as DDoS attacks and ensuring reliable performance for genuine users.
Name OptanonConsent
Provider .calendly.com
Retention period 1 year
Type Third party
Category Necessary
Description The OptanonConsent cookie determines whether the visitor has accepted the cookie consent box, ensuring that the consent box will not be presented again upon re-entry to the site. This cookie helps maintain the user's consent preferences and compliance with privacy regulations by storing information about the categories of cookies the user has consented to and preventing unnecessary repetition of consent requests.
Name OptanonAlertBoxClosed
Provider .calendly.com
Retention period 1 year
Type Third party
Category Necessary
Description The OptanonAlertBoxClosed cookie is set after visitors have seen a cookie information notice and, in some cases, only when they actively close the notice. It ensures that the cookie consent message is not shown again to the user, enhancing the user experience by preventing repetitive notifications. This cookie helps manage user preferences and ensures compliance with privacy regulations by recording when the notice has been acknowledged.
Name referrer_user_id
Provider .calendly.com
Retention period 14 days
Type Third party
Category Necessary
Description The referrer_user_id cookie is set by Calendly to support the booking functionality on the website. This cookie helps track the source of referrals to the booking page, enabling Calendly to attribute bookings accurately and enhance the user experience by streamlining the scheduling process. It assists in managing user sessions and preferences during the booking workflow, ensuring efficient and reliable operation.
Name _calendly_session
Provider .calendly.com
Retention period 21 days
Type Third party
Category Necessary
Description The _calendly_session cookie is set by Calendly, a meeting scheduling tool, to enable the meeting scheduler to function within the website. This cookie facilitates the scheduling process by maintaining session information, allowing visitors to book meetings and add events to their calendars seamlessly. It ensures that the scheduling workflow operates smoothly, providing a consistent and reliable user experience.
Name _gat_UA-*
Provider rubygarage.org
Retention period 1 minute
Type First party
Category Analytics
Description The _gat_UA-* cookie is a pattern type cookie set by Google Analytics, where the pattern element in the name contains the unique identity number of the Google Analytics account or website it relates to. This cookie is a variation of the _gat cookie and is used to throttle the request rate, limiting the amount of data collected by Google Analytics on high traffic websites. It helps manage the volume of data recorded, ensuring efficient performance and accurate analytics reporting.
Name _ga
Provider rubygarage.org
Retention period 1 year 1 month 4 days
Type First party
Category Analytics
Description The _ga cookie is set by Google Analytics to calculate visitor, session, and campaign data for the site's analytics reports. It helps track how users interact with the website, providing insights into site usage and performance.
Name _ga_*
Provider rubygarage.org
Retention period 1 year 1 month 4 days
Type First party
Category Analytics
Description The _ga_* cookie is set by Google Analytics to store and count page views on the website. This cookie helps track the number of visits and interactions with the website, providing valuable data for performance and user behavior analysis. It belongs to the analytics category and plays a crucial role in generating detailed usage reports for site optimization.
Name _gid
Provider rubygarage.org
Retention period 1 day
Type First party
Category Analytics
Description The _gid cookie is set by Google Analytics to store information about how visitors use a website and to create an analytics report on the website's performance. This cookie collects data on visitor behavior, including pages visited, duration of the visit, and interactions with the website, helping site owners understand and improve user experience. It is part of the analytics category and typically expires after 24 hours.
Name _dc_gtm_UA-*
Provider rubygarage.org
Retention period 1 minute
Type First party
Category Analytics
Description The _dc_gtm_UA-* cookie is set by Google Analytics to help load the Google Analytics script tag via Google Tag Manager. This cookie facilitates the efficient loading of analytics tools, ensuring that data on user behavior and website performance is accurately collected and reported. It is categorized under analytics and assists in the seamless integration and functioning of Google Analytics on the website.

How to Test UX Design: UX Problem Discovery, Hypothesis Validation & User Testing

  • 18354 views
  • 14 min
  • Feb 23, 2022
Elena S.

Elena S.

Writer

Oleksandra I.

Oleksandra I.

Head of Product Management Office

Share

Customer-driven product development implies designing and developing a product based on customer feedback. The match between actual product capabilities and end users’ expectations defines the success of any software project.

At RubyGarage, we create design mockups, wireframes, and prototypes to communicate assumptions regarding how your app should look and perform. User testing is then needed to validate these ideas before the actual product development starts. Why? Because early UX validation takes significantly less time and effort than rebuilding a ready-made product.

Learn how to test UX design ideas with this ultimate practical guide from RubyGarage UX experts.

What is UX validation?

In a broad sense, UX validation is the process of collecting evidence and learning around some idea with the aim of conducting experiments and user testing to make informed decisions about product design. You can validate a business idea, a user experience, or a specific problem with an existing product, or you can choose the most viable solution among all available options.

There are two approaches to UX validation:

  • Waterfall: Validation of the whole concept at the final release
  • Lean: Validation of individual hypotheses through multiple experiments
Waterfall vs Lean UX validation

Lean UX validation is preferable for startups due to lower risks of failure compared to the Waterfall approach and optimized budget distribution.

Why do you need UX validation?

The validation phase of UX research gives the product team an understanding of what the future (or existing) product should be like to satisfy the end user. It helps the team:

  • Understand customer value more profoundly. Get precise feedback on each feature and every blocker on the user’s way to conversion.
  • Align the product concept with user expectations. There’s no better way to form the correct product value for customers than listening to what real users want.
  • Find product–market fit. You need to be on the same page with your target audience to build a viable product that will bring desired business results.
  • Save budget and resources. Early validation per iteration helps reveal mismatches between features and customer expectations so you don’t spend time and money on the wrong features and UX flows. This way, you’ll invest resources with maximum efficiency.

You can use a lot of theory about UX validation methods to run the design validation cycle with your team. However, very few teams know how to test a UX design in practice. It’s rather challenging to organize all activities correctly and effectively. The RubyGarage UX design team has a well-thought-out UX validation workflow that we’ve polished through years of working on our own and clients’ products. Here is our practical step-by-step guide on running the UX validation process based on our in-house workflow.

Step 1: UX problem discovery

UX problem discovery involves researching the problem space. During this step, the team identifies the problems to be explored and solved after collecting evidence and determining what to do next.

1. Organizational preparation

The problem discovery process can differ depending on the project stage (whether a new or existing product is under development), the team structure, management, etc. However, here are the main preparation steps you should arrange before starting the UX validation process:

  1. Define the purpose and objectives for conducting UX validation. What product design flows do you need to validate and what UX problems do you need to test? We recommend using the Project Charter template to outline these items.
  2. Project Charter
  3. Define stakeholders to participate in the UX validation process. Identify each process participant and their areas of responsibility. We highly recommend elaborating stakeholders’ roles through the Stakeholders Register, Stakeholders Influence Matrix, or Stakeholders RACI Matrix.

Stakeholder register. contains contact information for each stakeholder such as their name, category, analysis, job title, and address. A stakeholder register may look approximately like this:

Stakeholder register

In the Stakeholder influence matrix, you can/should structure stakeholders by two essential criteria: power (influence) and interest. Power defines the impact of each stakeholder from the decision-making standpoint. The level of interest is how likely a stakeholder is to take action to exercise their power.

Stakeholder influence matrix

Stakeholder RACI matrix serves to identify who should be responsible, accountable, consulted, and informed:

Stakeholder RACI matrix

To structure the roles and responsibilities of stakeholders, use RACI ranging based on the following characteristics:

  • Responsible: Who will be doing the task?
  • Accountable: Who is responsible for making decisions? Who is going to approve it?
  • Consulted: Who can tell me about this task, activity, etc?
  • Informed: Who has to be kept informed about the progress? Whose work depends on this task, activity, etc?

As a result, you’ll get an outline of your UX validation process with all required participants defined and ready for the following activities.

2. Kick-off workshop meeting preparations

The kick-off workshop meeting is conducted to align the entire team around the challenge in order to form a UX validation plan with activities each team member (stakeholder) will perform.

Here is how to prepare for the kick-off workshop step by step:

  1. Define the workshop goals and participants. Define the workshop goals and activities to achieve those goals. Use the Stakeholder Register to outline the required participants.
  2. Prepare a kick-off workshop agenda to give participants a clear overview of the meeting activities, the duration of each step, and who will lead each task.
  3. Share the agenda with all participants before the workshop via established communication channels (email, Slack, etc.) to ensure they receive it.
  4. Create a workshop canvas to structurize and organize all activities and prepare the workspace for documenting the progress.

A workshop canvas is required so that the facilitator can effectively conduct the kick-off workshop meeting and gain the target results through the range of activities. At RubyGarage, we use the Miro whiteboard tool to map templates for all planned activities.

UX research template in Miro

3. Conducting a kick-off workshop meeting

Run the following activities during the kick-off workshop:

  1. Reframe the initial problem. Analyze the problem from three perspectives (desirability, feasibility, and viability) to more deeply understand the issues related to the briefing. We recommend using the Abstraction laddering tool. Put the problem statement in the middle. Go with ’why?’ questions up the ladder to get to the root cause of each challenge. Go down the ladder with ’how?’ to explore the issue more precisely and reveal sub-problems.
  2. Map existing knowledge and assumptions. Map out what you already know as a fact and what is still unknown in the context of discussed problems.
  3. Plan activities and pick tools to reach your objectives. Think about the practical activities to do in each discovery phase and pick those that are just enough to achieve your goals. Create a shared sprint plan that clearly defines the time frame for each activity, the participants responsible for each item, and how they will collaborate across roles. Put all milestone meetings, sprint ceremonies, and deadlines into a plan. Further, you’ll need to revisit this plan to track progress and adjust further steps if necessary.
  4. Fill in the after-meeting section in the agenda file to outline the meeting results and document all findings.
  5. Prepare a UX validation plan and approve it with the client and stakeholders. The final deliverable of the kick-off workshop meeting is a UX validation plan that includes the list of activities for the UX design validation process; the tasks backlog for each activity; and assignees, participants, and estimates for each task.

We recommend that you prepare a UX validation plan in the form of a presentation with clear descriptions for each activity, including its purpose, goals, execution, and deliverables. By doing so, it will be clear to the client and other decision-makers what you will do and why.

Step 2: User behavior analysis

During the kick-off meeting, you defined the UX problems that should be researched and validated. It’s essential to collect more data on how users currently interact with the product’s interface to understand these problems from the user’s point of view and prepare for user interviews. This is called behavioral analysis. This type of user testing helps you find answers to the following questions:

  • Where do users click within the product screens?
  • Where do users get stuck?
  • What screen areas and interface elements cause problems with the user experience?
  • How long does it take users from first click to conversion?
  • How can you nudge users to take actions?
  • What types of UI elements are most effective?

Analyzing behavior includes the following steps:

  1. Select analysis goals, KPIs, and metrics. Outline your plan for behavior analysis and pick relevant tools to accomplish your goals. Define the time frame and people responsible for conducting the research.
  2. Define user journeys for analysis. Users accomplish their actions via multiple scenarios through the product interface. Clearly outline user journeys for analysis to track the current activity and roadblocks. Map these user journeys in your UX validation canvas.
  3. Set up the required tools. Follow the selected tool setup and configuration guides to make them ready to track user activity.
  4. Set unique identifiers for each user to distinguish the activity of a specific user.
  5. Collect and analyze results. Based on recorded activity, define patterns in user behavior. Outline deviations in expected user behavior and define problem areas in the UX flow.
  6. Develop UX problem hypotheses. Formulate UX problems and define UX flows that must be validated during user interviews.

The deliverable at this step is a behavior analysis report to share results with the client and outline your findings for the next steps.

Tools for behavior analysis

Tools for behavior analysis

The right tool depends on the specifics of your product and your research subject. Different tools present user behavior statistics in different formats, from visualized heatmaps in Hotjar to statistical charts in Amplitude. Here is a short overview of the most popular user behavior testing tools:

  • Mixpanel: A powerful mobile app analytics tool that helps you collect and analyze data on specific actions and events set up in advance.
  • Hotjar: We prefer using Hotjar to identify UX-related problems causing user drop-offs. Hotjar provides heatmaps, session recordings, and surveys to understand customer behavior.
  • Heap: This tool doesn’t require upfront configuration. It automatically tracks each user’s activity and aggregates the collected data in various reports.
  • Amplitude: A comprehensive analytics tool to analyze user behavior with a view to multiple marketing metrics including retention, conversion rate, and lifetime value. 

Step 3: UX hypotheses validation

Over the previous steps, you got a set of UX hypotheses about what causes problems for your users, where they get stuck, what user flows to improve, and so on. You now need evidence to determine if these hypotheses are valid and worth further elaboration. There are different ways to test UX design hypotheses. At RubyGarage, we prefer in-depth interviews due to their informative value. Here is how our UX audit team approaches this step.

1. Select the right users for testing

Define who from the product’s customer base or target audience should participate in user interviews.

  • Define user personas. Create a typical description of each category of customer to be interviewed. Decide whether you need to interview people from various demographics, with different experience, etc. Document the defined personas.
  • Define sample quantity. Select how many users you need for each persona to receive enough representative information to validate hypotheses. It all depends on the type of your study, the variety of user patterns, and how people use your product. The Norman Nielsen Group recommends 40 participants for most quantitative studies to obtain statistically reliable results.
  • Recruit enough participants. You should find participants that match your target audience. You can get them from among your teammates and coworkers (internal recruiting), or you can look for suitable interviewees outside your company (external recruiting). If you engage participants internally, aim at employees not in the product team to get objective feedback. If you go for external training, focus on crowded places like malls or coffee shops to look for participants that represent your target audience. To speed up the recruitment process and find relevant participants for UX validation interviews, use specialized platforms like User Interviews or Ethnio. Both of these platforms offer an extensive database of vetted research participants with the ability to filter by multiple parameters.
  • Define methods of user allocation. Interviews can be conducted in person or remotely via phone, video call, or chat. Interviewees should be motivated to participate and provide high-quality insights.

When describing user personas, focus on their characteristics and experience using your product. Here is a user persona template for your reference:

User persona template

2. User interview design

Getting ready for user interviews takes some time. Having all visuals, questions, and forms prepared in advance ensures interviews will run smoothly and that participants will feel comfortable, guided, and engaged for effective interaction. When preparing for user interviews, focus on the following:

  • Collect user data about actual users’ experiences with your product
  • Collect insights into what users think about your product
  • Reveal real UX problems that users face
  • Define user pains, gains, wants, needs, and wishes related to the product’s UX solution

Keep in mind those criteria when preparing for interviews through the following steps:

  1. Prepare visual materials. Get a clickable prototype, working system version, or other visuals to use for your interviews. Define the strict order of showing visual materials and create a separate document with links to all visuals in the proper order.
  2. Prepare the interview structure. Determine the sequence of interview steps with the appropriate timing for each step.
  3. Prepare the user interview script. Define the questions for the interview in the proper sequence.
  4. Allocate the necessary number of users. Schedule the required number of interviewees and set up personal meetings depending on the selected allocation method (remote or in-house).
  5. Prepare a form for gathering feedback. This form must be filled out to sum up the findings after each interview session.

3. Running user interviews

Once you’ve created all required documents and scheduled meetings (or virtual sessions), begin running interviews:

  1. Conduct an interview with each participant. Go through the interview questions, following the set order and timing.
  2. Fill in all results in a feedback gathering form. Within one hour after each interview session, fill in the interview feedback form with your observations and information collected from the interviewee.
  3. Prepare a user interview report. Summarize your observations, insights, and data collected during user interview sessions in a report. Prepare analytical conclusions based on the collected data and share them with core stakeholders and the client. Define the most crucial insights gathered during the interview for the next UX validation activities.

One approach for processing user interview findings is Affinity mapping:

Affinity mapping approach to structure user interview responses

To follow this UX validation method:

  1. Record all notes or observations in a document (this can be a Google Document or a Miro board with sticky notes).
  2. Look for patterns in your observations and group them accordingly.
  3. Create a group for each pattern or theme.
  4. Give each group a name.

If you validate a range of hypotheses during user interviews, you can run a hypothesis-driven analysis combined with Affinity mapping, grouping the findings for and against specific hypotheses:

Hypothesis-driven user interview analysis

Step 4: Form a list of UX problem hypotheses

After the problem discovery and user testing phases, you can form a backlog of UX problem hypotheses. Based on this backlog, the UX design team should ideate solutions during the next steps of UX design validation. Formulate your hypothesis using the problem hypothesis framework:

Problem hypothesis framework

Each hypothesis should contain a proposed solution, the definition of success (a goal whose completion defines that the solution is successful), and evidence of your statement (facts and data collected during user behavior analysis and user testing).

Final thoughts

As a result of the first part of the UX design validation process, you get a defined list of UX problems and some hypotheses of how to solve them. Instead of vague ideas, you get grounded reasons for making improvements to your product’s UX design. The next step is generating potential solutions and choosing the most viable one for implementation. We uncover these steps in Practical Guide to UX Design Validation Part 2: Problem Definition and Solution Validation.

CONTENTS

FAQ

  1. UX design validation is essential when you develop the design for a new product or solve the usability problems in the existing solution. Determining the most suitable UX design approach before the implementation helps save time, budget, and team resources and prevent the product’s failure on delivery and release stages.

  2. Depending on the project complexity and scope of challenges, it may take up to a couple of weeks. Most of this time is spent organizing the required activities and analyzing the obtained results.

  3. You should define your target audience and recruit participants who match your customer personas. We recommend looking for suitable interviewees outside your product team to get objective feedback on the research questions. The best option is to use specialized platforms like User Interviews or Ethnio.

Authors:

Elena S.

Elena S.

Writer

Oleksandra I.

Oleksandra I.

Head of Product Management Office

Rate this article!

Nay
So-so
Not bad
Good
Wow
5 rating, average 5 out of 5

Share article with

Comments (0)

There are no comments yet

Leave a comment

Subscribe via email and know it all first!