Projective Exercises: Round Three
I've read Scott Taylor's response to my entry on projective exercises, and I agree with my friend and colleague on two points: (1) in the 25+ years we've been together, we've never had so much as a cross word, and (2) a difference of professional opinion does not equal cross words.
I have no intention of breaking our non-cross-words streak now. But when it comes to the substance of Scott's rebuttal, I remain firmly unconvinced.
For one thing, much of Scott's argument underscores what I consider to be a big flaw in the reasoning used by proponents of projective techniques -- an unquestioning, almost circular faith in their effectiveness. I personally need to see more evidence before I agree with statements like:
- "It's unlikely that consumer would have spoken of his/her pride in using the product in question without use of this projective technique."
- "Every time I have used a projective technique . . . I have come away with insight I couldn't have uncovered without use of such a technique."
Actually, I'll take this a step further. I continue to think projective techniques yield shallow, superficial responses at best, and red herrings at worst. Not to keep harping on that poor participant who pointed to the picture of the lion and explained I feel proud, but:
- To me, her response doesn't offer a lot of deep insight into the complex and nuanced factors that drive behavior. Pride may be a deeply felt emotion, but as a moderator (or if I were a product manager), I'd want to delve much, much deeper than that.
- I also worry that at best, the exercise may have encouraged this respondent to oversimplify what were probably more complex feelings. At worst, I fear her response may be misleading: she chose the lion picture because it was the only picture on the wall that at all resembled how she felt, but it's not exactly how she felt.
- And as moderators, do we really think we need to go through the trouble of setting up a projective exercise to get people to say something as general as "I feel proud"? In my experience, we can uncover broad emotions like pride (not to mention more complex, nuanced feelings) by building strong rapport with a participant and asking thoughtful, direct, probing questions.
As most everyone knows, psychologists sometimes use projective techniques in their attempts to tap into the underlying emotions of their patients. Unlike us dabblers, however, psychologists who use such techniques get extensive training in how to implement them and interpret the results (and based on the literature I've seen, interpretation is a lot more rigorous and involved than a lay person might think). But even with rigorous application and interpretation, projective techniques in psychology have been roundly criticized as unreliable and invalid.
I did a quick Internet search, and it didn't take more than a few minutes to find journal articles outlining the problems with the use of projective techniques in psychology, including one from Scientific American in 2004. Those interested can read it here: http://www.psychologicalscience.org/pdf/pspi/sa1_2.pdf. Those who prefer to have their faith in projective techniques unshaken may not want to read the piece, however, because among its conclusions is the following:
| "We find it troubling that psychologists commonly administer projective instruments in situations for which their value has not been well established by multiple studies. . . . Our results . . . offer a broader lesson for practicing clinicians, psychology students, and the public at large: even seasoned professionals can be fooled by their intuitions and their faith in tools that lack strong evidence of effectiveness. When a substantial body of research demonstrates that old intuitions are wrong, it is time to adopt new ways of thinking." |
I'm happy that we can keep this discussion at a professional, respectful level. On second thought, I don't just feel happy -- I feel . . . proud.
For related entries, see the following:
Round I -- Peter Fondulas: Eggy-Weggs
Round II -- Scott Taylor: Eggy Weggs? Put Up Your Dukes!
http://www.thetaylorgroup.com/blog/trackback.cfm?019BC73F-1422-22E7-23D478148339B763
I should begin by saying that in the eight years I have known Peter Fondulas we have never had a cross word either, and that is entirely due to Peter's remarkable composure and restraint. Scott and I have had a few cross words, but I think -- and hope -- we have put those professional disagreements behind us.
When A Clockwork Orange was released, Rex Reed, the brilliant film critic for the New Yorker referred to it as "the only perfect movie I have ever seen." I feel much the same way, as I don't believe there has ever been a film which more clearly defined what was wrong with our society. A Clockwork Orange decried manipulative leaders, scientists, administrators and parents who want to fix symptoms not problems, and to benefit themselves rather than society.
So when our hero and humble narrator Alex talks about wanting to smash the "Eggy Weggs" he has in fact been returned to his old self, for better or worse. The initial "fix" that was employed served to turn Alex into little more than an automaton who had no real freedom of choice. Even though the symbolism of using eggs (read: start of life) has some horrific implications as to Alex's newly re-found perception of the world, he is at least being honest.
How does this all relate to market research? For one thing, I would much rather hear from the Alex who is pathologically sinister than the one who had been "fixed" -- simply because you get honest answers in a way the fixed Alex could never provide.
Now jumping into the fray about projective techniques. Sorry Peter, but this is an exercise that I absolutely LOVE.
I began using projective techniques extensively while working for Disney Channel years ago. By using food and cars as the example, we were able to establish why Disney Channel's brand image among kids needed to change -- and just how starkly it differed from that of competitors Nickelodeon and Cartoon Network. The focus groups we conducted help justify a re-launch of Disney Channel in 1996. One could say we knew the answer already, but when a room full of ten executives has it thrown in their faces, the result is coldly sobering.
-- An interesting sidebar to these kids focus groups. The moderator we employed was a woman we had never used before, but came recommended as an expert on children's issues. When going around the room to ask what car they perceived Disney Channel as, one kid said a "washing machine." Instead of probing as to why he gave this answer, the moderator turned to lecture the kid to go back and do the exercise correctly. A huge groan went up in the back room as the moderator missed what might have been a "aha!" moment. If I had a cane available I would have yanked her off stage at that moment. Needless to say we never used that moderator ever again.
When done correctly, projective techniques allow the respondent to separate themselves from logic and use creativity. Some people are naturally more imaginative than others. But as a client, what we take away from this exercise is the ability to move around the company from that day forward and remind executives of these comments. And not just because they are so powerful and so evocative, but also because they are so easy to remember! And Scott's point about making executives squirm in the back room can not be over-emphasized in its importance. Sometimes you do need to hit people over the head to get their attention, and this is often the first step towards curing the problem.
I can not release anything publicly right now, but the results of our own brand personification exercises conducted a while ago are actively being utilized in formulating our present strategies. They came directly out of focus group research Scott conducted and were of enormous value to us. I can not over-state the usefulness of this research here. It was critical because it provided confirmation to senior management that our hypothesis was correct -- but it also provided the framework for allowing our marketing and advertising people to fully hone their ideas. What is equally important here is to be able to use this technique over time and to see how the brand imagery has changed, something that is critically important to our business, especially with the launch of new competitors.
I do recognize some may think this tool to be a little trite and worn. But through creative use of imagery and proper implementation, this is still a very effective exercise. And do these, as Peter suggests, yield only shallow results? On the surface I can see where one might conclude that. But this where extensive probing comes into play, because it is of course less important that a respondent compares a brand with say, a Cadillac, than it is to understand what a Cadillac really means to the respondent and why they chose it. Does the Cadillac example lack the robustness needed to begin this exploration? Given the enormous potential to probe into how the Cadillac (and the brand) relate to the respondent's background and experiences, I'd have to say it's as good a place as any to start.
As another sidebar, my wife of ten years is a psychologist. I know that she uses a variation of symbolism and projective techniques in conducting dream analysis with her patients. While some criticize this method, she and many colleagues in her field maintain that working with the unconscious is the only way to effectively treat people.
And yes to the answer the question you are thinking, I do get free psychotherapy sessions, although they usually arise unsolicited.
Back to A Clockwork Orange. As our hero Alex concludes the movie by assuring us his mind (or "Gulliver" as he so creatively put it) had been restored back to normal, I only hope Peter you take another look at this exercise. Projective techniques allow us as researchers to glean honest responses in their most raw form. And just as importantly, they allow end users of the research the opportunity to take actions they might not otherwise have considered.
Finally, I've been searching my brain for favorite Alex quotes from that movie, and the one that always seem to spring to mind is as follows. I don't think it refers to market research, but it may come in useful one day:
"It's not my fault, sir. I was led on by the treachery of others!"