While the failures of industrial-scale algorithms are often attributed to some failure of machine learning engineering, many of these failures actually stem from something else entirely: the human beings whose behavior generates the data used to build these algorithms. For example, research shows that prejudice can arise not just from preferences and beliefs, but also from the way people choose. When people behave automatically, biases creep in: quick, snap decisions are typically more prejudiced than slow, deliberate ones, and can lead to behaviors that users themselves do not want or intend. As a result, algorithms trained on automatic behaviors can misunderstand the prejudice of users: the more automatic the behavior, the greater the error.

Bio:

Diag Davenport is a presidential postdoctoral research fellow at the Princeton School of Public and International Affairs, where he studies various topics at the intersection of big data and behavioral economics.