Decisions are such an essential part of life. We make decisions in almost every sphere of life.
Data-driven decisions are those decisions that are backed by data rather than intuition or rules. There is a massive amount of data that is being generated and getting a one fit rule for all that data is close to impossible. So the rules for making the decisions are derived using data analytics.
In the healthcare domain, the decisions made have a large number of stakeholders; people, their life, their family and their livelihood.
So it's imperative to reduce the error to a minimum while making those decisions. In the forthcoming example, a healthcare provider in the US shifted to Big data and data science to analyse the data and find out which population of people are prone to a kind of lifestyle disease. Let's hear from Utkalika as she walks us through that case.
For shifting to Big Data to solve that problem, what the health provider needed was a massive amount of data. So they started collecting the medical records, history of medicines, the amount of money they spent on medications previously. After receiving the data, they contacted a third-party vendor to clean the data and build a classification model on it. They identified the right population of people who were prone to that illness. Now, after implementing the model, they expected to cut expenses and make more money. So after months, they analysed the data to find out how the model helped them achieve their objective. What do you think they found out?
When they analysed the data, they found out that they had missed an important parameter, that is race, and the algorithm assumed that it didn't matter the race you belonged to. It was later identified that the target population group that the algorithm predicted was simply the one that had incurred higher medical expenses because those people could afford medical care. So, as a result, the people who were already well to do had access to even better medical and healthcare facilities. The algorithm was written by a vendor who was known for writing medical algorithms, which meant that the same algorithm was being used in multiple healthcare facilities, which made the problem worse.
So, even if data-driven decisions can save millions of dollars for a company, even a small bais like this can result in compromising the life and well-being of the entire population.