Enterprise resource planning systems¶
Design feedback sessions¶
Applied methods & measures
UX research methods: qualitative interview studies, prototypes as stimuli, discussion guides, thematic analysis, collaborative analysis
Measures: UMUX lite, SUS, single ease question
Goal: Identify user needs for customers doing specific accounting tasks.
Approach: Design feedback sessions were conducted in the discovery or exploratory phase to collect (a) contextual information about a participant's job role and duties that help to understand current work practices and (b) obtain feedback on a specific design/flow using a prototype as a stimulus. Depending upon the prototype's fidelity, the participants did interact with the prototype themselves or it was a mixture of moderator-guided and participant-driven exploration.
Findings/Results (due to confidentiality, only an outline is provided):
- Uncovered needs that participants expressed (explicitly and implicitly) which impacted the product design (feature design & roadmap)
- Confirming design choices that resulted in implemented product features
- Challenging the product and design team's assumptions resulting in feature changes/reprioritizations
- Systematically documenting the real-world processes and challenges the customers are facing
Role of Joachim Pfister:
Executing on all phases of the UX research process.
UX research challenges successfully addressed
- Creating an interview environment for participants to feel comfortable with (ice-breaker questions, sharing their work experiences before moving into the design feedback part, which they didn't know what was waiting for them)
- Designing and executing research activities that creatively address internal and external challenges
- Defining the target population and criteria for the research activity
- Recruiting the participants (done either by myself or with assistance from research ops)
- Holding optional debrief sessions involving all observers/stakeholders which helps to identify common themes quickly
- Keeping stakeholders in the loop by providing session summaries and, if permitted, sharing the video recordings with them
Concept validation¶
Applied methods & measures
UX research methods: qualitative interview studies, prototypes as stimuli, discussion guides, thematic analysis, collaborative analysis
Measures: UMUX lite
Goal: Check early design ideas or product concepts with customers.
Approach: For conducting discovery and exploratory research, the concept validation sessions were designed in a way to quickly immerse a participant (doing a specific accounting task) into a new or different way of approaching their established task. This was a truly collaborative effort involving the researcher, designer and product manager to obtain visuals like storyboards and compelling narratives. This thorough preparation helped to align the product team on the concept itself. In consequence, we narrowed down and sharpened the customer-facing questions as well.
Findings/Results (due to confidentiality, only an outline is provided):
- Participants were readily challenging the concept's approach and the embedded ideas using their real-world knowledge and experiences.
- Uncovered needs that participants expressed (explicitly and implicitly) which impacted the product design (feature design & roadmap)
- Confirming assumptions by the product team that subsequently were used to design features
- Challenging the product and design team's assumptions resulting in concept changes and generating new ideas
Role of Joachim Pfister:
Executing on all phases of the UX research process.
UX research challenges successfully addressed
- Creating an interview environment for participants to feel comfortable with (ice-breaker questions, sharing their work experiences before moving into the concept validation part, which they didn't know what was waiting for them)
- Strengthening the whole product team's understanding of the concept by making it presentable and understandable to a customer
- Designing and executing research activities that creatively address internal and external challenges
- Defining the target population and criteria for the research activity
- Recruiting the participants (done by myself or with assistance from research ops)
- Holding optional debrief sessions involving all observers/stakeholders which helps to identify common themes quickly
- Keeping stakeholders in the loop by providing session summaries and, if permitted, sharing the video recordings
Customer visits & contextual interviews¶
Applied methods & measures
UX research methods: interviews, contextual inquiry, card sorting
Measures: internally defined measures
Goal: Empathize with customers in their real-world work environment when they were doing their job. For some customer visits, a wider spectrum of other NetSuite job roles (such as engineers, designers, technical writers, etc.) joined those "field trips" in order to gain first-hand experience how customers are using the product they are working on.
Approach: A customer visit starts with doing an intro and setting expectations for this type of "learning visit" for all the participants from the hosting customer company and the visiting NetSuite employees. Customer participants were either observed doing their daily job or they walked the NetSuite observers through their most common tasks. No dedicated interview script was used but notes were taken diligently to capture the customer participant's feedback and their pain points. A final debrief session with all participants included a game-based activity to collect feedback playfully and wrap up the visit. The NetSuite team debriefed afterwards and did a collaborative analysis and findings were shared with the respective product teams and in presentations.
Findings/Results (due to confidentiality, only an outline is provided):
- Assessing customer needs and their pain points through observation in their real-world work environment
- Uncovering usage patterns that would have not been otherwise discovered
- Helping other job roles to create empathy with the customers/users
- Understanding the whole service delivery process from the customer's perspective beyond isolated research activities which will benefit future design activities
Role of Joachim Pfister:
Executing on all phases of the UX research process.
UX research challenges successfully addressed
- Setting customer-expectations what the visitors as observers will do
- Engaging team members that are not involved with research activities and exposing them to some methods and the benefits of talking to customers
Surveys¶
Applied methods & measures
UX research methods: quantitative methods, surveys
Measures: internally defined measures
Goal: Identify common preferences and common approaches for the larger-scale customer base.
Approach: An initial draft of a survey was produced by me as the researcher to discuss it with the other product team members and stakeholders. The questions were heavily inspired by previous, qualitative research elements originating from other research activities. For me, a survey is a tool to validate previously collected, qualitative-based assumptions as hypotheses. After collecting the potential survey questions, the questions were prioritized (e.g. which ones are key for design) and the flow and wording were discussed with other researchers. Depending upon the number of responses, the time available, and the goals associated with the research, either simple descriptive statistics were used (e.g. creating insights & visuals with Excel) or more complex statistical analyses (involving R) were used.
Findings/Results (due to confidentiality, only an outline is provided):
The results helped to understand, for example, which elements of a dashboard were preferred by the customers. Those data points were used as input for product roadmap decisions.
Role of Joachim Pfister:
Executing on all phases of the UX research process.
UX research challenges successfully addressed
- Choosing the appropriate methods for analysis (consulting with other researchers)
- Using R libraries to quickly create visualizations for visual data analysis
Unmoderated studies¶
Applied methods & measures
UX research methods: unmoderated studies
Measures: UMUX Lite, CSAT, Single ease question
Goal: Piloting unmoderated studies for the product area with company-internal participants.
Approach: One research activity proved to be the perfect fit for conducting an unmoderated research study with NetSuite-internal participants. First, the research goals were defined. Then, in close collaboration with the designer, the clickable research prototype was developed alongside the tasks that needed to be achieved. I set up the unmoderated study in UserZoom and recruited the participants. Since this was a pilot for an unmoderated study, the research activity included also some feedback questions on the methodology itself (e.g., what went well or if the participants were preferring a moderated research session). Findings were compiled in reports and presentations (on the product-focused research activity and the unmoderated approach itself.)
Findings/Results (due to confidentiality, only an outline is provided):
Unmoderated studies were highly appreciated by the participants due to the flexibility offered when they can conduct them on their own.
Role of Joachim Pfister:
Executing on all phases of the UX research process.
UX research challenges successfully addressed
- Documenting and sharing the experiences with other researchers after piloting this research method in the product area
- Identifying and observing organization-internal policies when internal employees were used as participants