Data Mining Emotions
Did anyone ever tell you that you wear your heart on your sleeve? It’s a popular expression, but
obviously no one is looking at your sleeve to read your emotions. Instead, we tend to study a
person’s facial expressions to “read” their emotions. Most of us think we’re rather good at reading
faces, but we couldn’t say exactly how we make our interpretations, and we don’t know whether
they are accurate. But what if we could use technology to know how another person is feeling?
Would it be ethical to do so in the workplace and then act on our findings?
Thankfully, technology is not quite ready to do this. Face reading is a complex science. Paul
Eckman, a noted psychologist, may be the best human face reader in the world. He has been
studying the interpretation of emotions for over 40 years and developed a catalog of over 5,000
muscle movements and their emotional content. His work even spawned a television series called
Lie to Me, in which the main characters analyzed microexpressions —expressions that occur in the
fraction of a second—to assist in corporate and governmental investigations. Using Eckman’s
Facial Coding System, technology firms like Emotient Inc. have been developing algorithms to
match microexpressions to emotions. These organizations are currently looking for patterns of
microexpressions that might predict behavior.
Honda, P&G, Coca-Cola, and Unilever have tried the technology to identify the reactions to new
products, with mixed results. For one thing, since expressions can change instantly, it is challenging
to discern which emotions prevail. A person watching a commercial, for instance, may smile,
furrow his brow, and raise his eyebrows all in the space of 30 seconds, indicating expressiveness,
confusion, and surprise in turn. Second, it is difficult to know whether a person will act upon these
