Consumer Centric

UK company FreshMinds Research recently ran a test by pulling social media commentary about Starbucks using several popular analytic tools offering automated sentiment analysis of the text gathered.  They found flipping a coin to determine the sentiment of each individual comment would have been more accurate than what the tools reported.

FreshMinds analyzed over 19,000 online conversations with tools from Alterian, Biz360, Brandwatch, Nielsen, Radian6, Scoutlabs and Sysomos.  All content was centered on Starbucks.

The good news is aggregate level reporting of sentiment (average overall) was between 60% and 80% in agreement with a manual coding by trained staff.  Not bad.  The bad news?  Only about a third of individual comments were accurately coded.

Somehow, the randomization of automation errors resulted in an aggregate number of coding all conversations that wasn’t off by much.  But, if you wanted to dig deeper into individual conversations either for more insight or to engage in the conversation, the likelihood of finding the right positive or negative comments is not very high at all.

Their report is an excellent overview of these seven tools and how they perform across geographies and content sources.  And, as a side note, it’s a great marketing effort to get you and me to pull down their paper in exchange for contact information.

It’s not surprising to me that these tools are still so far off.  It’s a micro-representation of a macro-level challenge facing most research firms, agencies, and marketers today:  putting things into context from a people-centric approach.  We have so much data today that making it both accurate and actionable requires a more concerted effort to put everything into context, mirroring the reality of human decision-making and behavior as much as possible.

I’m sure some combination of neural networks, complexity science, and/or agent-based simulation tools eventually will yield “smarter” sentiment analysis tools to speed up the process of sifting through thousands of lines of text-based data.  Those pursuing that dream need not lose sight of the biggest mystery to solve:  understanding the meaning of words within a human context.

The FreshMinds report is definitely worth the read.  I’m curious what the makers of these tools would have to say about their report.

Thanks to Research (the magazine) for the heads up on the white paper release.

I came across a few online research platforms this morning that are new to me.  It sort of spread to a little hunting expedition, which led to a few more interesting discoveries.  Here’s a snapshot of my quick read on a few.

Revelation and QualVu were both launched in 2007.  They stand out to me, however, because they’re legit social media-style platforms, but built the way a researcher would think and want to use the information.

Revelation appears to be the next generation of blog, chat room and bulletin board type research tools.

QualVu is a very robust online video sharing platform.  Their VideoDiary product is a complete solution for easily facilitating face-to-face and/or personal video diary-stye feedback from real people in their real environments.

I can’t wait to try them both out.  I’ve heard QualVu has been used by a lot of people.  I’ve still not found a Revelation user yet.  I’d love to get real user feedback.

Of course, this little discovery led to a chase down of a few other interesting tools worth checking out.  One I already bought a subscription to this morning after finding it.

First, CiviCom has it’s own set of 3-D virtual community solutions for marketing research.  I’m not totally certain on the nature of this company in terms of being built by researchers for researchers.  But the idea of helping to facilitate interactive collaboration in virtual environments is very cool.

My favorite find of the day (or the one I really hope works as I’ve bought an initial month subscription to give it a shot) is GuapoVideo.  What I love about this is it is built for internal audiences to collectively annotate and analyze video gathered through research.  Brilliant idea — a tool to help make internal collaboration and co-creation even easier.  It’s like a video-editing suite built from a researcher point of view to upload, annotate, and then cut and past clips into your presentations or share via a web page they create for you.

I also wonder if GuapoVideo could also be used as a way to reach back to the person(s) you interviewed and have them respond to your interpretation of their comments in a more co-creative way.  An added-value step in a hybrid research or ethnographic project.

A funny, but practical one, too.  Ask500People is a totally different online feedback tool, but one brand managers, account planners, and strategists could use in a pinch to get some basic feedback on concepts, ideas, etc.  While I take issue with the way they represent margin of error for polls on their site (I wouldn’t call this scientific sampling for the market research purests by any stretch of the imagination), the concept of a quick question feedback tool is an example of easy, collaborative methods for getting others to think about the issues you’re consider.

Finally, in my brief field trip to discover interesting online research tools I came across this informative slideshare presentation by Carol Phillips of Brand Amplitude.

Share more you’ve come across that have been helpful.

Sitting in church recently I couldn’t help getting caught up in the energy of the baby boy sitting on his dad’s lap just in front of us. I wasn’t the only one–my wife and kids joined me in making weird faces and waving to the little guy for nearly an hour! You know exactly what I mean if you’ve ever made eye contact with a cute tiny one who stares back at you with genuine intrigue.

During this back and forth, I was captivated by the interaction and real communication between this boy and his very attentive dad. The kid couldn’t have been more than nine months old; he was barely sitting up with the balancing act facing you when you’re head is bigger than the rest of your torso. He couldn’t speak. He couldn’t write. But boy could he ever communicate. And he and his dad had a very real conversation throughout the church service. Dad had to work hard, but he delivered at every stage of the little guy’s experience at church that morning.

So, what’s the point? And why was I thinking about work ideas at church?

The kid’s happiness and enjoyment were a direct result of dad’s interest in him, familiarity with him, and relentless responsiveness to what he was ’saying’ to dad; all of this without a single word being spoken.

Consider this interchange. Dad sees smile on kid’s face, dad smiles back, dad picks up string of beads and drapes it on kid’s ear. Kid desperately tries to get his hand to find and grasp the beads only to knock them off his ear and fall to the ground. Dad picks them up and the scene is repeated until kid’s attention shifts to his sister. You get the idea. You’ve seen it before. This was repeated in various iterations with food, toys, fingers, etc.

Dad listened. Dad’s genuine interest put him in a position to understand every emotional shift expressed on his son’s face. Dad was prepared with a seemingly endless stash of tactics to test in an attempt to meet his son’s needs. They interacted. And it worked because he observed, understood, and acted to a certain extent on the kid’s terms. Dad accomplished his goal of entertaining his son during the church service.

Companies with genuine interest in their customers, who know their customers, and who relentlessly respond to what they observe succeed in helping the people they interact with find happiness. Sure, we call it satisfaction, loyalty, or passion about a brand. In the end, isn’t it really about caring enough, doing enough, and interacting enough to act on their terms? The terms of your customers.

In my view, that is the point of being people centric. Companies who get it will plan and deliver their marketing (the relentless interaction trying to solve a kid’s needs) and create their products (trying out from an endless stash of toys and things) to be truly authentic and relevant.

Word of mouth within this context is real, because actions and behaviors are real.