DIY User Interview Tips for Product Managers

Modern product managers know that they must speak with their users regularly. I started following this maxim diligently many years ago. However, far too frequently the thrill of speaking to a customer often trumped the value I got from it. I am glad to say that as time has passed, I feel I have gotten better at talking to users.  

Helped by the fortunate opportunity to shadow a trained and brilliant qualitative research expert for a significant time, and forced by circumstances to depend solely on customer conversations to overcome adoption barriers, I have devised certain methods and guidelines which help me approach user interactions with a definite goal and emerge from the exercise with specific inputs. 

I am sharing below some of the steps and guidelines I adopt for myself, before speaking with users. 

Know what you want to learn? 

I have found it very helpful to be specific about the questions that I want answers to. Rather than approaching a user conversation looking to get a ‘sense of’ what they think about my product, I now have a specific learning goal in mind, before starting the process.  

The following are some examples of the questions that I have sought answers to: 

• How do sports fans consume content on their phones? 
• Are users able to understand how my app solves their problem X? 

• Why are people signing up but not making use of the free trial? 
• What are the goals and aspirations of 24–28 years old Indian men with a disposable income less than 20K a month? 

 Test a hypothesis 

Another important part of beginning the exercise is to form some kind of hypotheses about the answers you are looking for, and then using the conversations to test whether your hypotheses are valid or not. 

For example, while talking to users of a sports app, one of the hypotheses I wanted to test was that users were getting put off by the long ‘intros’ we had at the beginning of video clips, and this was a key factor in them not watching the rest of the video. I was then able to plan my complete user interaction to test whether my hypothesis was correct or not. 

 A few deep conversations are worth way more than many shallow ones 

When I am setting up time for a user conversation, I try and find a time slot that allows for the possibility of the conversation continuing far longer than planned. When the user knows you are interested, they usually don’t mind talking to you for hours!  

When it comes to quantitative research, I am often mindful of the sampling process and sample size. However, with user interviews, I have found that even half a dozen deep conversations reveal enough useful information to keep your ‘action items’ list populated for a while. 

A contextual inquiry expert once told me that you need five deep conversations to start identifying patterns and refining your hypotheses, five more to validate them, and maybe an additional five to talk about something new that you may have thought of during those ten conversations. Over the last decade or so, my personal experience has shown this to be a very good thumb rule. 

Ask users what they did, not what they would do 

At a SaaS startup I was once involved with, one of the founders loved to ask each prospect, “How much will you pay for this product?” He may have asked this question to hundreds of potential customers and received hundreds of different answers. So, when it was time to price the product, he did not have a clue! 

The trouble with asking someone what they would do or what they would want, is to allow them to choose an answer based on parameters that may not apply when it actually comes to displaying that behavior.  

I would rather ask my user to name the colors of the shoes they own, than ask if they would buy shiny golden ones. I would ask them about the paid apps that they have already downloaded on their phone, rather than ask them if they would pay for mine. I prefer to ask users what they did when they logged in to the app the last time, rather than ask them to name their favorite feature! 

Understand the context of use 

One of the best lines of questioning that reveals why a user did what they did, is to ask them about exactly where they were, who they were with, and what they were doing at the time of using a product, or experiencing a part of the user journey. 

For example, sports fans revealed how they would take a sneak peek into their phones to check the score during meetings. In a certain region, newly married men living in joint families repeatedly mentioned that they bought contraceptives only when away on holiday with their spouses, which almost always was to a religious destination. Similarly, many smokers living with their parents consume mouth fresheners (or even toothpaste) every time they expect to encounter their parents after they smoke. 

Give them a task and observe how they do it 

While testing whether users were able to navigate through one of India’s best known OTT apps, we asked users to find a video of Virat Kohli reaching his century in the previous day’s match. The product team had embedded the most significant highlights from the match within the scorecard, and called it the Video Scoreboard. Unfortunately, users never expected to find videos in a scorecard, did not interpret the name in the way the product team expected them to, and often failed to find the video even after five minutes of searching. 

Giving users a task that the product is supposed to enable, and observing them go about the task can tell you a lot about what features work and which ones do not. It can also provide great insights into how users go about a task, and what could be the intuitive way to solve their problem.  

After you understand the what, ask about the why? 

One of my early mentors from the field of user experience design often spoke about the relationship between thoughts, beliefs, feelings and actions, and the impact of this relationship on product design. 

I have been part of research exercises where experts have culled out the thoughts, beliefs and feelings driving a particular user action. Without the requisite knowledge and training I have found it impossible to achieve that level of clarity when I have interviewed users myself. 

However, I discovered that asking specific questions to understand why users think they did what they did, can also be very helpful. For example, accounting professionals who were wowed by some automation features we had built, refused to use our product in spite of being impressed by some of its capabilities. After speaking to some of them, we found the reason – our target users all ‘believed’ that they could complete the task faster than our software.  

Don’t miss the wood for the trees while interpreting your findings 

Talking to multiple users, especially in the context of an existing product, can often leave you with a long list of feature requirements. Product managers are often left confused about what to make of the overwhelming number of suggestions, and which ones to prioritize. 

However, drawing the right conclusions from a series of interviews is a skill in itself. Conducting the whole exercise as a hypothesis testing project is a way to ensure that you are not drawn to conclusions you did not set about looking for. 

The most important point to keep in mind however, is to focus on patterns and responses that are common across users, not their individual solutions to solving problems. For example, when many small business owners told me that they would start using my cash flow management software if it had this feature, or that feature, the easiest conclusion for me to draw was that my current solution was not fulfilling the needs of any of them.

Conclusion 

All PMs are encouraged, and indeed must make time to speak with their users. Many PMs are forced to do it anyway as their organizations don’t have the resources to hire experts for the job. In either situation, the exercise needs to be conducted with the goal of maximizing the benefits from these interactions. I have listed above some of the approaches that have helped me get better at talking to users, than I was maybe a decade ago. If you have developed your methods as well, please do share them with us in the comments.