Questions for a Researcher — In Interviews and at Work

Recently, I came across an interesting 2016 article that discusses interview questions for UX researchers applying to FAANG companies. Some of these questions really resonated with me.
For example:
Imagine that you recommended fixing a usability issue, but a developer responded, “The metrics data from millions of users tells us that this issue doesn’t exist.” How would you respond?
This is a fantastic question. It illustrates a situation that any researcher might encounter, and it also has a hidden layer, a twist. It perfectly highlights a critical issue in our industry and an important contradiction.
Indeed, how can analytics data prove the absence of a usability issue? Especially when we observe participants encountering it during tests. What kind of data could refute this? A lack of user drop-offs on this screen? Perhaps the speed at which users navigate through this screen doesn’t slow down compared to other pages? Maybe there were pre-redesign metrics for comparison.
To what extent should a researcher be confident in their insights and recommendations? Can they explain to the developer the difference between proving presence and proving absence? Must a usability issue always be visible in metrics? How would the mechanism of triangulating qualitative and quantitative data work to refute it? The more you ponder the answer, the more questions arise. Honestly, I’m thrilled by this.
I believe the genre of interview questions is a goldmine. It’s by resolving such scenarios that we grow as researchers and as an industry. Asking such questions prompts us to reflect, turning a research lens onto researchers themselves.
I decided to try writing a few more questions with twists. Let’s try answering them together:
Arkady wants to conduct a quantitative survey of app users to measure UMUX. The product owner suggested recruiting an equal number of experienced and inexperienced users to enhance representativeness. How valid is this suggestion?
Irina is struggling to choose a research method. On the one hand, she wants to conduct in-depth interviews. But the product manager insists on running a custdev. How should she navigate this tricky situation?
Over the course of a week, the team gathered hypotheses for Nikolay to validate. Most of them were added to the research backlog, but one keeps bothering him: “The app will allow sorting news by the number of likes from readers.” How would you test this hypothesis if you were Nikolay?
During a usability test of an image-viewing app, Prokofy suddenly realized that some participants didn’t understand how the system worked. For the next test, he invited a developer to explain the framework’s intricacies to users. What would you have done in Prokofy’s place?
Gennady has been writing a report on five problem-solving interviews for two weeks. He’s gathered a massive amount of material, and the presentation already has 150 slides. He’s currently on the fifth slide about why participant #3 will never use the product—after all, the participant clearly stated they didn’t like the app. How can Gennady finish the report faster?
Mikhail gave a brilliant presentation on the usability test findings. The issues of critical severity were glaring, and the designer cried after watching the user recordings. However, it turned out that the feature being studied had already been removed from the product. This happened around the time Mikhail started conducting the first interview. What can Mikhail do next time to avoid a similar situation?
How would you answer these questions? What question would you ask to highlight an issue that matters most to you in the UX research industry?