All associates at Taylor worth their salt know Scott's focus group intro by heart:

"Nothing you say here tonight will come back to haunt you" always loosens participants up a bit.

"You cannot give a wrong answer." (This is very true.)

And my favorite: "I hope you will stick to your own opinions, whatever those opinions may be."

That last line, which is so important inside the focus group room, is equally significant for those observing in the backroom.

One of the reasons qualitative work is so rich is it gives observers the chance not only to hear in-depth responses to questions, ideas, and stimuli, but also to watch body language and facial expressions, while listening for tone and inflection.

These very benefits are also what can lead to contradicting opinions on findings between researchers.

Recently, Scott and I found ourselves in that very position. Scott moderated two groups on a television pilot. I watched the groups the next day. We wrote up our key findings independently. After reviewing Scott's findings, I saw that although for the most part we were on the same page, there were a few instances in which we simply did not agree. Where Scott had interpreted tones, body language, or replies from respondents to mean one thing, I had taken them to mean something different.

We discussed our differences in opinion and reviewed the supporting materials we had collected (respondent notes, DVDs of groups, and notes transcribed from these DVDs). Eventually we agreed upon a resolution and reported findings that had the most evidence to support them.

Differences in opinion are bound to happen from time to time, as the nature of qualitative research is largely subjective. One researcher's opinion could very well contradict another's -- one reason why some market researchers prefer quantitative work, with numbers that provide hard data to crunch.

However, qualitative work provides added depth to research that otherwise would be lost. Examining contradictory opinions of participants who are very often screened to have similar demographics is valuable and intriguing.

And this got me thinking: There's something poetic about two market researchers who are exposed to exactly the same stimuli (in this case, focus groups) and take from it something different, even contradictory. I wonder, should someone out there be conducting qualitative research . . . on us researchers?

Fantasy Football drafts are only a couple of months away. Weren't Fantasy Baseball drafts just three months ago? Yes, they were -- I know this because I hear all about it from my husband and his equally excited male friends.

It seems it's a big deal to them.

It's an extremely small deal to me and to my girlfriends. Actually, it's not even on our radars. And that got me thinking.

I love my husband beyond measure, but I will never completely understand him. In fact, that goes for all males. This goes deeper than their fascination with Fantasy sports or their ability to spend hours on a golf course. This isn't even about their vast knowledge of cars or their inability to remember our friends' names.

What I really want to know is: What is going on up there? What makes them tick?

Men say women are confusing, but I beg to differ. Women are easy! We like to talk -- about our joys, our problems, or anything, really. When I ask my husband what he is thinking, he tells me, "Nothing, really." When he asks me what I'm thinking, I will not only tell him what's on my mind, but I will also tell him why I think I'm thinking what I'm thinking, what I was thinking that led me to my current thought, and how these thoughts make me feel.

In our qualitative work, we often split groups by age (younger vs. older), by status (consumers vs. prospects), and yes, by gender.

Recently I was in the backroom for a group where company X was being discussed. A woman relayed her experience with the company using feelings. She spoke of how she felt when company X didn't meet her standards, how customer service for company X made her feel valued as a customer, and how this feeling of value was the ultimate reason she stayed with the company (even though she still had some unresolved issues).

A man in the same group (for this project, respondents were not split by gender) relayed his experience with company X in facts. This happened, then this, then this. There was an issue; he called customer service; they fixed it.

I've seen this scenario quite a few times while observing in the backroom. One of the things I find most fascinating about our qualitative research is how differently males and females process events.

And I've often wondered: Since men and women are so different, can we really make observations when comparing an all-male group and an all-female group? There are such deep differences in the thought processes of the genders that your learning from each group is almost always very different. Might it be better to always mix genders in groups (at least when the research calls for groups with both genders), and avoid all single-gender groups?

What do you think?

Despite my unapproachable nature, I occasionally meet new people. And they always ask me: What do you do? (This question, while not particularly specific, has come to be understood as an inquiry about one's profession.)

So I repeat what Scott Taylor once told me that he told people who asked such questions: that we're a market research firm that helps large organizations better understand their customers and potential customers, that we obtain evidence and analyze it, that we work in industries like entertainment and technology and pharmaceuticals and many others.

And before they completely tune me out, I tell them that half of our work consists of focus groups.

Everyone has heard of focus groups (often in a not-so-positive light, like in politics, where a "focus grouped candidate" is used as a pejorative description), so sometimes that sparks a reply. I tell them focus groups are basically in-depth conversations with a small number of targeted respondents, to gain insight into the reasons behind people's opinions.

And then it becomes clear I've talked enough, as their inquiry was a polite conversation-starter and I've gone on too long. So I return the favor: And you?

But there's more to say about focus groups -- a lot more about the art of moderating, different techniques and approaches, and the pursuit of a successful group.

In an attempt to capture some of this, we've put together a short video of our moderators talking about their craft. It's available here or by clicking the button on our homepage.

Check it out. Soon we'll release extended one-on-one interviews with our moderators; those will also be available on our website.

And coming this summer, for the number-crunchers: a quantitative video!

This weekend I read George Will's eulogy for William F. Buckley, Jr., the famous conservative voice of the 20th century (founder of National Review). Will noted the following: "One clue to Bill's talent for friendship surely is his fondness for this thought of Harold Nicolson's: 'Only one person in a thousand is a bore, and he is interesting because he is one person in a thousand.'"

If anyone were looking for a defining characteristic of a really good interviewer/moderator, you couldn't find a much better one than this.

Earlier this month, two clients contacted us about moderating focus groups in Boston.

Why is this notable? Well, we almost never conduct qualitative research in Boston, despite being headquartered 50 miles to the north.

In 2007, Boston barely finished among our top-10 cities for qualitative research:

• 1. New York
• 2. Washington, D.C.
• 3. Atlanta
• 4. Philadelphia
• 5. Southwestern Connecticut
• 6. Los Angeles
• 7. Dallas
• 8. Chicago
• 9. Phoenix
• 10. Boston

When looking to conduct focus groups and IDIs in a Northeast market, our clients opt for New York, Connecticut, Washington, D.C., and Philadelphia. But rarely do they choose Boston.

Considering the healthy number of well-educated, higher-income consumers that most of our clients are targeting, I wonder: Why don't our clients, many of them headquartered in New York, want to conduct research in Boston?

A few possibilities:

It's too white -- and, at the same time, ethnic. Boston's metropolitan area is more than 20% whiter than the national average, and has a reputation of being more segregated than other major U.S. cities. Boston residents are thought to have a stronger-than-average sense of cultural identity, which could set them apart from potential participants in other cities.

It's too educated. Boston Public Schools, the oldest public school system in the country, is still considered the nation's best, and the array of colleges and universities provide a population that could be seen as too intellectual.

It's weird and isn't representative of the U.S. A lot of people think New Englanders are just strange and don't represent the rest of the country. (Many New Englanders appreciate this reputation.)

What do you think?

For every study, we work with our client to identify the goals and then map out a gameplan to achieve them.

Most of the time, everything goes according to plan.

Sometimes it doesn't. Sometimes the study is going smoothly, the recruit is great, and then we get to the focus group facility -- and they throw us a curveball. For example:

A bad experience: Many years ago, we did a series of focus groups in a strip mall in New Jersey. Our project team showed up to find that the participant room had a jury-rigged table made up of two smaller tables pushed together. Of course, the tables didn't match, giving the whole thing a thrown-together, unprofessional look.

Then they served dinner to the clients: stale white-bread sandwiches. They were horrible.

And to top it off, according to one of our colleagues, the hostess spent all night ignoring her responsibilities and flirting with him (warning to the reader: this part of the story may be apocryphal).

A funny experience: We once spent two days/nights at a facility where the well-meaning host was so overly attentive that he was stalker-like. Every few minutes he'd come in the backroom to make sure the project team was okay and didn't need anything.

He stationed himself outside the door of the focus group room, and every time someone would come out to go into the back (to see if clients had questions), he'd immediately ask, "Is everything okay? Do you need anything?"

After the first night of groups, as our vice president started to put on his jacket to leave, the host ran over, took the jacket, and helped him put it on. Then, toward the end of the last group on the final night, the VP came out of the room to check in the back and jokingly said to the host, "Now comes Miller time."

He went back into the group to finish up, and this time when he walked out, the host was standing there, trying to shove a bottle of beer into his hand.

An awkward coincidence: Recently we were doing groups in Stamford, Connecticut. As the first group entered the room, one of our clients exclaimed, "That guy looks exactly like Steve (real name withheld)." A moment later another client started walking slowly up to the window/mirror and shouted, "Hey! That is Steve!"

As it turned out, Steve worked for our client, and his presence there was not appreciated. Naturally, we immediately had him pulled from the group, but he was invited to watch the groups in the backroom.

He stayed for 20 minutes and left.

Surely you've had similar experiences. Feel free to share them.

UPR: Unconditional Positive Regard. That's what they taught me in focus group moderator training, and I try to live by that principle in every group I moderate. In fact, I love all my focus group participants. I really do.

There are just some I love a little less than others.

Do any of these folks sound familiar to you?

The First-Talker: Maybe my least favorite participant -- er, the participant I have the most "less love" for. This participant is always the very first to open his mouth, after every single question. Special moderating tactics seem to work only temporarily (looking away from FT when you ask the question, standing behind him, prefacing questions with "let's hear from someone on this side of the table on this one"). I know I'm sunk when the whole phenomenon turns rhythmic (question, FT responds, others respond . . . question, FT responds, others respond . . . lather, rinse, repeat). You know you've reached that point when you ask a question, FT hesitates, and everyone else in the group stares at him, not saying a word until he weighs in.

The Spokesman: Then there's the participant who takes on the task of speaking for everyone in the room, or summarizing "what we're all saying." One of my favorite variations was the participant I had in a study testing positioning statements. Following qualitative research protocol, I was careful to distribute each statement on a worksheet, to elicit private, uninfluenced reactions. Mr. Spokesman looked down at the statement I had just passed around and declared, "Anyone who likes this statement is an idiot." (Needless to say, that particular positioning statement was not a huge hit in that session.)

The Topic Expert: This is the person we typically think of as the "dominator" of a focus group, the person who has some history or expertise with the topic. Good screening fortunately keeps this a relatively rare occurrence, but former lives can come back to haunt you. The one-time graphic design expert in a group testing new magazine layouts. The former medical professional in a group testing claims for a new medication. As with the First-Talker, this can also lead to that sinking experience of watching all eyes turn toward this participant whenever you ask a question.

The Marketing Expert: These are participants who always tend to answer for the "marketplace" or the "masses," which, interestingly enough, usually does not include themselves.

• "Which of these ideas is most appealing?"
• "I think you should go with option A."
• "Is that the one you personally like best?"
• "Me? Oh no, I don't like it at all. But I'm sure it would be the most profitable."

The Long-Winded Responder: I don't know about you, but in many of my groups I've got a jam-packed discussion guide, and I cherish every one of those 120 minutes available to me. So when a participant takes 5+ meandering minutes to make a point, and 5+ meandering minutes to make the next point, and the next point, I find myself unable to concentrate on anything but the numerals on my digital clock. And if that person also happens to be the First-Talker, I know I'm in for a long two hours.

The Clueless Wonder: Almost more of a source of amusement than annoyance is the participant whose responses always seem to miss the point. Every time CW opens his or her mouth, you can almost feel the "huh?" emanating from the backroom.

The Silent One: The classic "there for the paycheck" participant, wondering if he or she can possibly get through the entire group without saying a word. Aha, I know, I'll call on him! "Sam, what's your opinion?" "I agree with what everyone else is saying."

The Texter: You wonder why Lucy keeps looking down during the session, until you realize she's texting someone with her cellphone. I'm always tempted to bring out my inner elementary school teacher -- "Lucy, would you like to share that with the rest of the focus group?"

The Spreadsheet-Buster: Okay, I don't really mind this person, but I can't help bringing it up -- the one person in every group who insists on offering a "3.5" rating on a 1-5 scale.

I'm sure I've left off some of your own "a little less loved" participants. Feel free to add to the list.

It was 30 minutes into my very long 90-minute focus group session when I turned around and noticed something strange: a disembodied head at the table.

Was it a focus group with gun enthusiasts gone horribly wrong? No. It was a focus group with young boys . . . and swivel chairs.

Extra coordination with facilities is always necessary when you are conducting focus groups with children (in this case, 9- to 11-year-olds), as more guidelines are needed to eliminate as many potential distractions as possible. This facility had followed our instructions to a tee: no soda or high-sugar foods were served to the kids, no food or drink was brought into the groups, and no items besides paper and pencils were in the room. Earlier, the girls' 9-11 group didn't even notice the swivel chairs, so we did not think to change them for the boys' group.

The first indicator of our oversight was when one of the boys decided to treat his chair like a Sit 'N Spin, spinning around and around. (I loved that toy.) The boy next to him also began the dizzying motion. When giggling ensued between the two boys, I immediately announced, "Okay, new rule: no spinning our chairs." Soon after this proclamation, the blonde boy sitting at the end of the table thought it would be the perfect time to test an amazing new chair feature he discovered -- the pumping mechanism.

After a minute of watching him play with the magic chair, I told him he needed to choose one level and stay there. He followed my instructions for a while, but at one point, when I turned around to ask him a question, I realized he had set the chair level so low that his chin was now sitting on the table. I had been talking to four kids and a head. He was probably just resting from his sugar high, most likely induced by his Coke and Lucky Charms breakfast.

I continued as if everything were normal, because he was still paying attention and, for some reason, none of the boys seemed to notice -- there was no giggling and no one else had decided to take the same plunge (literally). Since it didn't seem to be distracting anyone, including the head, I played along.

The head eventually became a real live boy again, and I made a permanent mental note to add another guideline to the list of facility requests for future kids groups.

I've read Scott Taylor's response to my entry on projective exercises, and I agree with my friend and colleague on two points: (1) in the 25+ years we've been together, we've never had so much as a cross word, and (2) a difference of professional opinion does not equal cross words.

I have no intention of breaking our non-cross-words streak now. But when it comes to the substance of Scott's rebuttal, I remain firmly unconvinced.

For one thing, much of Scott's argument underscores what I consider to be a big flaw in the reasoning used by proponents of projective techniques -- an unquestioning, almost circular faith in their effectiveness. I personally need to see more evidence before I agree with statements like:

• "It's unlikely that consumer would have spoken of his/her pride in using the product in question without use of this projective technique."
• "Every time I have used a projective technique . . . I have come away with insight I couldn't have uncovered without use of such a technique."

Statements like these beg the question, How do we know we could not have elicited similar reactions without projective techniques? Where, aside from "conventional wisdom," is the evidence?

Actually, I'll take this a step further. I continue to think projective techniques yield shallow, superficial responses at best, and red herrings at worst. Not to keep harping on that poor participant who pointed to the picture of the lion and explained I feel proud, but:

• To me, her response doesn't offer a lot of deep insight into the complex and nuanced factors that drive behavior. Pride may be a deeply felt emotion, but as a moderator (or if I were a product manager), I'd want to delve much, much deeper than that.
• I also worry that at best, the exercise may have encouraged this respondent to oversimplify what were probably more complex feelings. At worst, I fear her response may be misleading: she chose the lion picture because it was the only picture on the wall that at all resembled how she felt, but it's not exactly how she felt.
• And as moderators, do we really think we need to go through the trouble of setting up a projective exercise to get people to say something as general as "I feel proud"? In my experience, we can uncover broad emotions like pride (not to mention more complex, nuanced feelings) by building strong rapport with a participant and asking thoughtful, direct, probing questions.

Some might say I'm missing the purpose of projectives -- that their true value is in offering useful starting points for further probing. But are the "starting points" robust and reliable? I'm not convinced. And even if they are, I'm confident we can get to the same, if not more insightful, starting points through other means.

As most everyone knows, psychologists sometimes use projective techniques in their attempts to tap into the underlying emotions of their patients. Unlike us dabblers, however, psychologists who use such techniques get extensive training in how to implement them and interpret the results (and based on the literature I've seen, interpretation is a lot more rigorous and involved than a lay person might think). But even with rigorous application and interpretation, projective techniques in psychology have been roundly criticized as unreliable and invalid.

I did a quick Internet search, and it didn't take more than a few minutes to find journal articles outlining the problems with the use of projective techniques in psychology, including one from Scientific American in 2004. Those interested can read it here: http://www.psychologicalscience.org/pdf/pspi/sa1_2.pdf. Those who prefer to have their faith in projective techniques unshaken may not want to read the piece, however, because among its conclusions is the following:

"We find it troubling that psychologists commonly administer projective instruments in situations for which their value has not been well established by multiple studies. . . . Our results . . . offer a broader lesson for practicing clinicians, psychology students, and the public at large: even seasoned professionals can be fooled by their intuitions and their faith in tools that lack strong evidence of effectiveness. When a substantial body of research demonstrates that old intuitions are wrong, it is time to adopt new ways of thinking."

I'm happy that we can keep this discussion at a professional, respectful level. On second thought, I don't just feel happy -- I feel . . . proud.

For related entries, see the following:

Round I -- Peter Fondulas: Eggy-Weggs

Round II -- Scott Taylor: Eggy Weggs? Put Up Your Dukes!

Peter Fondulas and I have been together for more than 25 years now, and never has a cross word passed between us. As a point of comparison, my wife and I have been together for 38 years, and several cross words have passed between us.

Then I read Peter's May 15 blog entry in which he argues that projective techniques in research "help me feel 'creative' . . . , [but] in and of themselves, they yield little more than broad generalities. . . . Quite frankly, I have my doubts about [their effectiveness]."

Fightin' words.

Blasphemy.

Where have you gone, Joe DiMaggio?

The goal in many of our projects is to identify the underlying emotions and images that motivate consumer behavior. But expressing image and emotion for a product, a service, or a brand isn't easy for consumers. Projective techniques employ analogy and metaphor ("If brand X were an automobile . . . ") to make it easier for consumers to think about and express their product or brand images and associated emotions.

In my experience, far from producing little more than broad generalities, projective techniques produce laser-like illumination of what we are trying to understand for our client. At the very least, such techniques create a compelling visual image of a brand for management to keep in mind in developing marketing strategy; often, such techniques provide a veritable blueprint for management action.

Peter's own examples are evidence for my point:

• It's true that in every brand personification exercise one brand is old and stodgy and wears a business suit, while another is young and hip and wears khakis; but I guarantee you the senior managers of the stodgy brand, sitting behind the one-way mirror, squirm a bit, shake their heads at what has become of their brand, and commit themselves to taking action. There's no future in being old and stodgy.

• In a photo sort when a consumer says, "I picked the photo of a lion because I feel proud," it may be cliché to us as researchers, because we hear such reactions all the time, but it's unlikely that consumer would have spoken of his/her pride in using the product in question without use of this projective technique. Pride, after all, is a deeply felt emotion; you can bet the brand managers behind the glass made note of how their brand makes its users feel--and how, by extension, they might position their brand to tap that emotion.

Every time I have used a projective technique--from color chips, to brand as automobile or animal, to the techniques Peter cites--I have come away with insight I couldn't have uncovered without use of such a technique. And, every time, our client has come away with a compelling visual image of its brand against the competition in the hearts and minds of consumers.

Yes, it's true--in a projective exercise we establish the context for consumers: think of the brand as a person (not a car this time, or an animal, or a color, or a sound, etc.). But that doesn't limit consumers' thinking; it opens their thinking. It allows them to think about something difficult to express (their emotional connection to a brand) using an analogy that provides a bridge from the familiar to the difficult.

"A good analogy," as psychologist Steven Pinker wrote in last Sunday's New York Times Book Review (reviewing Natalie Angier's The Canon), "does not just invoke some chance resemblance between the thing being explained and the thing introduced to explain it. It capitalizes on a deep similarity between the principles that govern the two things."

Projective techniques capitalize precisely on this notion of "deep similarities." What may seem like "cliché and generality" for the seasoned research professional (because we hear the same reactions over and over to, for example, the brand personification exercise) is not cliché and generality at all--it's evidence of archetypal beliefs and emotions that reveal themselves precisely through the use of these various projective techniques.

I just now reread Peter's May 15 entry. What was it that little kid said to Shoeless Joe Jackson (he of the 1919 Black Sox scandal in baseball)?

I remember.

"Say it ain't so, Joe. Say it ain't so!"

[NOTE TO READER]: A difference of professional opinion does not equal cross words.