Company Uses Facial Recognition to Assess Learning

Hong Kong-based Pathship had an interesting idea about learning that led to an even more interesting idea about analytics. The learning idea was that, with today’s technology, one can efficiently match up knowledge seekers to knowledge providers (e.g. novices to experts) for one-on-one conversations. Because we now live in a world where embedded analytics are de rigueur, this led to the question of how to assess whether the one-on-one conversation enhanced learning.

Since the training world began there has been only one reliably practical answer to measuring the value of a conversation: Ask the learner to rate it. Now there’s a new answer: Use computer vision to assess the attentiveness of the learner. If they look bored then they’re probably not learning; if they look engaged then they probably are.

This “science fiction” technology is now here thanks to computer vision (see this paper by Joseph Grafsgaard et al.). Given that these knowledge seeker-provider calls would normally be done by video conferencing, all you need to do is plug the computer vision tool into the video stream and ‘boom’ you’ve got measurement.

If you think about this for a moment you’ll recognize that this computer vision measurement of attentiveness could be used for any e-learning — as long as the learner has the camera turned on. It could not only tell us in general whether a course (or conversation) engaged a learner, we could tell which parts were interesting and which were boring.

From there we might imagine using cameras in meetings to assess attentiveness and give us a warning sign if we were boring our staff to death. We might use the technique in sales training or customer service work (again presuming we can get the prospect or customer on camera) to see how well we are keeping them engaged.

Analytics loves data and it loves it best if that data can be gathered automatically. Computer vision that recognizes attentiveness and other emotions offers a potentially astounding tool for automatically gathering data in a way we could hardly imagine before.

Special thanks to our community of practice for these insights. The community is a group of leading organizations that meets monthly to discuss analytics and evidence-based decision making in the real world. If you’re interested in moving down the path towards a more effective approach to people analytics, then email me at dcreelman@creelmanresearch.com