Understanding Human Factors Studies: A Comprehensive Guide Beyond Formative and Summative Validation

By Margarita Cajina, Senior Human Factors Engineer, Interface Analysis Associates LLC

Key Insights

• Human factors research supports product safety and usability well beyond formative and summative validation studies.
• Different types of human factors studies align with different stages of the product development lifecycle, from ideation through post-design validation.
• Engaging human factors early helps identify usability risks when they are less costly and easier to mitigate.
• Selecting the right study at the right time strengthens design decisions, regulatory readiness, and real-world product performance.

---

Product development is a constant back-and-forth endeavor of ingenuity and compromise. With regulatory requirements, engineering constraints, business goals, and tight deadlines always at the forefront, it is easy for one of the most critical elements to get lost: the people who will actually use the product. A product can be innovative and captivating, but if the intended user is confused by it or unable to use it safely, the product’s benefits are severely undermined.

There are many ways to ensure that the user remains a focus during product development, with human factors testing being one of the most well‑known methodologies. While many product sponsors are aware of formative and summative (validation) studies as they relate to satisfying the Food and Drug Administration’s (FDA’s) regulatory process, fewer understand the importance of engaging human factors early and often in the product development cycle.

Many are also unaware of the broader range of user research methods that can be applied to creating a safe, effective, and usable healthcare product or system, as reflected in FDA guidance on human factors and usability engineering.

This guide outlines common types of human factors studies used throughout the product development lifecycle. Study types are organized by the stage they most directly support, recognizing that many can be applied across multiple phases.

Ideation

Ethnographic Study
Ethnography involves researchers observing users in their natural environment. The goal is to identify problems or opportunities to improve user processes that may not yet be explicitly labeled as “problems” or “issues” because users have adapted to them or created workarounds. These studies are intended to identify gaps in the market that competitors may have missed. Ethnography can be conducted with current users of the same or similar products, or with prospective users who have not yet experienced the product but can reveal real-life constraints that may affect product use.

Interviews
The purpose of interviews is to understand users’ motivations for doing things the way they do and to capture their opinions as they relate to the product or environment of use. Interviews allow for the collection of data that may not be directly observed or measured, such as users’ thoughts, perceptions, and feelings.

Concept Development

Prototype Study
In the early stages, prototype studies can use low-fidelity prototypes and screen mockups to determine whether a concept makes sense and whether users understand the workflow or process steps. Incorporating this type of study early can save time and money by identifying necessary changes to the user interface before development progresses too far.

Handling Study
When a product has a physical user interface, an early-stage handling study can be used to assess whether users can physically manipulate the product and use it as intended. These studies also allow researchers to observe users’ natural tendencies when positioning or manipulating the product.

The prototype should be weighted to be as close as possible to the intended product design and include representative shapes, buttons, and features to ensure accurate data collection. Handling studies can influence product shape, size, force requirements, and textures to ensure the product is as ergonomic as possible.

A/B Testing
In A/B testing, two or more design options are evaluated to determine which is more effective, user-friendly, and aligned with users’ mental models (heuristics) and physical capabilities.

This approach allows teams to present multiple concepts to users and determine the best design direction or assess the impact of a potential design change. A/B testing can also help determine whether a change is too subtle or too significant, potentially causing user confusion.

It is important to minimize third-variable influences to reduce noise in the data and ensure core research questions are answered effectively.

Marketing Objectives

Competitive Comparison Study
In this study type, early concepts are tested against existing products on the market. The goal is to objectively measure performance, usability, and users’ perceived value relative to competitor offerings. Results are evaluated to help define technical and commercial requirements and to inform early design decisions.

Design Validation and Iteration

Formative Studies
Formative studies are intended to identify usability strengths and weaknesses early in development and allow for modifications that improve design. These studies may include different use scenarios, contexts, and product labeling elements, such as Instructions for Use.

It is highly recommended to plan time after a formative study for product iteration and, when appropriate, additional formative studies. This approach helps confirm that design changes effectively mitigate potential use errors and minimize use-related risk. Addressing issues early leads to a more seamless user experience once the product is finalized.

Some specialized types of formative studies include pre-clinical, design optimization, and pilot summative studies:

• Pre-Clinical: Used to evaluate the product in preparation for a clinical trial. Conducting human factors testing before a clinical trial helps minimize the impact of use-related errors and design flaws on clinical data. A user-friendly product can also increase dosing or treatment compliance. Consideration should be given to the training provided by clinical staff and tailoring Instructions for Use to reflect the clinical setting.
  
• Design Optimization: Used to generate ideas for improving product design. These studies typically involve a small number of participants who provide detailed opinions and feedback about the product’s design and use experience, which can then be used to identify opportunities for refinement.
  
• Pilot Summative: Conducted as a final formative step before a summative (validation) study. Pilot summative studies use the final product and methods identical or similar to those planned for the summative study. These studies help reduce sponsor risk and cost by identifying potential issues before a full summative study is conducted.

Summative (Validation) Study

This is the final study in the product development process to ensure that the final product design—including the device or system, Instructions for Use, and supporting materials—supports intended users’ safe and effective use under the intended conditions of use. Summative studies are also intended to validate that use‑related risks have been eliminated or mitigated as much as reasonably possible.

For products requiring approval by the Food and Drug Administration (FDA), the results of a summative study are submitted to the FDA for review. In the case of combination drug–device products, validation studies play a particularly critical role in demonstrating that the user interface supports safe and effective use without introducing new use‑related risks.

Additional considerations for successful human factors validation are discussed in Shifting the Paradigm: Four Key Principles for Successful Human Factors Validation for Combination Drug and Medical Devices.

Labeling Differentiation Study
Labeling differentiation studies evaluate whether users can distinguish the proposed product from comparator products based on packaging and device labeling. These studies consider factors such as physical appearance, color schemes, and labeling elements to ensure clear differentiation.

Self-Selection Study
Self-selection studies are conducted for over-the-counter products to determine whether users understand labeling, contraindications, and if they can safely select the product based on their specific health situations. The primary objective is to assess whether users can correctly determine if the product is appropriate for them.

Comparative Use Human Factors (CUHF) Study
CUHF studies assess differences between a proposed generic drug-device combination product and its Reference Listed Drug (RLD). The goal is to demonstrate that user interface differences do not negatively impact safety, effectiveness, or the use-related risk profile, and that the generic product is suitable for substitution with the RLD.

Conclusion

There are numerous opportunities to evaluate and refine a product throughout the development process through various forms of user research and testing. By incorporating human factors research early in design, many safety and usability issues can be identified when they are less costly to address. The goal is to prioritize error prevention and user experience by selecting the right studies at the right time, resulting in products that go beyond functionality and aesthetics and support successful use in real-world contexts.

To learn more about Interface Analysis Associates LLC’s human factors services, please contact us.