Don’t forget the microphone!

Speech signals are very rich in information about a person’s emotional state. You can listen to a computer’s microphone and monitor the audio from the user; even if they do not talk per se, but grunt and grumble, there are still useful signals in the voice data which you can use to gauge emotional state. My own company created an API to do just this, and on the main demo page, you can in fact record a snippet of audio right from your browser and see the underlying emotional state. The site is at


I was responding to a Quora post today about sentiment analysis, and I realized just why I created Moodzle: 

I personally think that with the advent of services like Siri, our culture is returning to orality, moreso than literacy. We will spend more and more time interacting through voice with our devices (and homes, and cars…) and for this, I believe we need an API which listens to audio to gauge the sentiment. 
We will spend less and less time reading, and more and more time talking. But we will expect our devices to be semi-sentient, and understand not only what we say, but why we said it.

Customer Service / American Airlines

I was on the phone yesterday with American Airlines because they canceled a flight I was supposed to be on, and I was slogging through the hassle of rebooking the flight. It was maddening to have to work through the automated voice interface to get to the agent I needed to speak to. No, I did not want to upgrade, no, I did not want to check my frequent flyer miles. I got pretty tired of talking to the automatic phone system. I was pretty frustrated with the system, and knew that no computerized system would be able to negotiate the rebooking for me, and for that, I needed to speak to an agent. By the time the agent came on the phone, I was audibly frustrated. Testy. Terse. 

But here’s the thing: I was already terse and testy throughout the experience of using my voice to navigate the phone system. If American Airlines was smart, they would have detected the emotional tone of my voice as I was wading through the phone system, so when I finally spoke to the agent, he would be better-prepared to deal with me. For example, given that I was in a testy mode, he could have been more conciliatory towards me off the bat, and that would have not only made my own experience better, but it would have likely made the whole call faster, which would have resulted in cost savings for American Airlines.

All they would have had to do was call EmotionAPI with my voice samples, while I was going through the phone system, and they could have gotten my emotional state by the time I finally spoke to an agent; by thing this into their booking system, the agent would have seen my state pop up on his screen, or perhaps the call would have gone into a different queue — such as a cal center in the United States, rather than an overseas call center. Small touches like this help improve customer loyalty, and yes, of course, help the bottom line too! And for mere pennies, this information is easy to get out of the EmotionAPI.