Machine Learning Models are Whistleblowers of Unethical Practices of Society

With the advent of the data science practises, especially the likes of Artificial Intelligence (AI) or machine learning and their adaptation in sensitive areas, has drawn massive attention to this technology in the recent years. Whether it is cancer diagnosing system, Amazon’s AI recruiting tool or the US court’s profiling system Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), we can always disagree to the decision made by the AI systems. In fact, it is our confirmation bias that leads us to agree or disagreeing of certain argument or decision, according to Hugo Mercier and Dan Sperber, author of the 2017 book “The Enigma of Reason”.

Originally published at LinkedIn Pulse on 11 January 2020.

Alex Knight @ www.pexels.com

Over 100 science fiction movies have been produced between 1927 and 2019 on AI, of which the majority has depicted AI in the form of evil killer robots or agents of death. Hollywood placed in our subconscious mind, scenes demonstrating how more intelligent machines will take over the world and enslave or totally wipe humanity from existence. The potential for AIs to become greater than human intelligence portrays a dark future for humanity.

While the concept of AI is around for quite some time—since 1956 to be exact—the technology only came out of its infancy and started to crawl like a toddler in recent years. Merriam-Webster defines AI as the capability of a machine to imitate intelligent human behaviour. The key here is the ‘mimicry’. The system doesn’t have to be intelligent itself as long as it can imitate intelligent behaviour.

The fuel for this technology is ‘data’, and is not limited to what we could store in a database or in an Excel spreadsheet. With big data technology, we can now process zettabytes of data, in the form of text, audio and video. History books, city council records, textbooks, magazines, encyclopedia, web sites, Wikipedia all are becoming the source of truth when it comes to training a machine learning model. Machines are now analysing and finding patterns in our past, the way we never even thought was possible!

These new findings came as a shocker to many. Eyebrows have been raised; questions have been asked to the validity of the AI model. Blames have been placed upon the AI model for being discriminative. Better scrutiny and explainability have been demanded the AI models. But is the technology really to be blamed?

Let’s have a look at the 2014 Stop-Question-and-Frisk program data (45,787 observations and 101 variables) from the New York City Police Department by which police officers stop and question hundreds of thousands of pedestrians annually, and frisk them for weapons and other contraband.

Number of Stops

Ratio of Stop and Population

Now, it is quite easy to infer from this data that for whatever reasons officers of New York City Police Department is more likely to stop black people more often, despite the fact that they are not the majority of the population in NYC. And now if we train an AI model and use that as the brain for ‘RoboCop’, what could we expect?

The bias in data doesn’t diminish the necessity of making the model bias-free. In fact, now that we know such bias exists in our data, we need to ensure to remove such bias or discard the data altogether (if necessary) before training our machine learning models. Troubleshoot ideas before committing to them. It’s better to find and fix vulnerabilities now—even if it means taking longer—than to have regulators find them later on. As a good practice, the following three keys need to be observed while training a machine learning model.

1. Select the right modelling technique for the problem.

2. Select training data, that is free from bias and represent every group included in the model.

3. Use real data to train, test and monitor model performance.

Therefore, feedback-loop based human judgement is needed to ensure the AI-based decision system is fair.