Bias in AI and Mathematical Models

Every single system is biased. Anyone who says otherwise is lying to themselves. My reasoning: every system is biased because every system is built by humans, and whether or not the mathematician or computer scientist is willing to admit it, every human is biased. That means that every system is biased. In reading the first part of “Weapons of Math Destruction” by Cathy O’Neil, I am absolutely astonished to realize that I have never once thought about bias in AI at all. Math has always been my safe space, a place where there is a right answer and many wrong ones. A place where I did not have to worry about bias from my teacher as long as my numbers added up. This thinking is not only untrue, but harmful. If people whole-heartedly trust all mathematical models, then we are simply turning a blind eye while the models take advantage of us and a lot of society. This quote from Weapons of Math Destruction puts it clearly:

“The math-powered applications powering the data economy were based on choices made by fallible human beings. Some of these choices were no doubt made with the best intentions. Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives” (Cathy O’Neil, Pg. 3).

This is a clear-cut quote from a magnificent author/mathematician that is not afraid to shy away from the hard subjects. The more I think about this book and this author the more the corruption makes sense. One example of how bias affects me daily is the lack of women in technology. Algorithms very much determine what we see and hear. The ads we scroll past on our phones, the emails we receive from colleges. I honestly do not think I have ever seen a promotion for a degree in computer science or math. I am a women in technology, and I am largely a minority in my field. This can sometimes feel incredibly isolating in classes as I look around the room and see just how much I am outnumbered. There are definitely many reasons to put the blame for the inequity in the technology industry, but I definitely put some of the blame on mathematical models/ AI — and their encoded bias.