Research on Researchers?

All associates at Taylor worth their salt know Scott's focus group intro by heart:

"Nothing you say here tonight will come back to haunt you" always loosens participants up a bit.

"You cannot give a wrong answer." (This is very true.)

And my favorite: "I hope you will stick to your own opinions, whatever those opinions may be."

That last line, which is so important inside the focus group room, is equally significant for those observing in the backroom.

One of the reasons qualitative work is so rich is it gives observers the chance not only to hear in-depth responses to questions, ideas, and stimuli, but also to watch body language and facial expressions, while listening for tone and inflection.

These very benefits are also what can lead to contradicting opinions on findings between researchers.

Recently, Scott and I found ourselves in that very position. Scott moderated two groups on a television pilot. I watched the groups the next day. We wrote up our key findings independently. After reviewing Scott's findings, I saw that although for the most part we were on the same page, there were a few instances in which we simply did not agree. Where Scott had interpreted tones, body language, or replies from respondents to mean one thing, I had taken them to mean something different.

We discussed our differences in opinion and reviewed the supporting materials we had collected (respondent notes, DVDs of groups, and notes transcribed from these DVDs). Eventually we agreed upon a resolution and reported findings that had the most evidence to support them.

Differences in opinion are bound to happen from time to time, as the nature of qualitative research is largely subjective. One researcher's opinion could very well contradict another's -- one reason why some market researchers prefer quantitative work, with numbers that provide hard data to crunch.

However, qualitative work provides added depth to research that otherwise would be lost. Examining contradictory opinions of participants who are very often screened to have similar demographics is valuable and intriguing.

And this got me thinking: There's something poetic about two market researchers who are exposed to exactly the same stimuli (in this case, focus groups) and take from it something different, even contradictory. I wonder, should someone out there be conducting qualitative research . . . on us researchers?