Machine Learning: Your Right to Explainability

Counting Sand

English - January 11, 2022 11:00 - 31 minutes - 29 MB - ★★★★★ - 23 ratings
Technology Business Careers artificial intelligence computer science machine learning big data systems thinking data science neural networks deep learning Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed

Previous Episode: The Boundaries of Personal Data

Next Episode: Bonus: Season 1 Recap

How do we make the next generation of machine learning models that are explainable? How do you start finding new kinds of models that might be explainable? Where do you even start thinking about that process from a research perspective?

Nikos begins with a discussion on how we make decisions in general. In the scientific world, we mostly reason through statistical or cause-and-effect type scenarios. We can predict outcomes and train our models to produce the results we traditionally expect.

He then discusses other early pioneers in this work, for example, back in the 70s, a rules engine was developed to help clinicians make diagnoses. It turns out that humans are very complex and hard to codify. Dr. Charles Forgy wrote his thesis on the Rete algorithm which is what modern-day rules-based engines stem from.

After the AI winter period, there was the introduction of neural networks that would encode the rules. This became an issue for explainability on why the rule was created. The neural networks create a mathematical weighted data model evaluated against the outcome. Without the ability to open up the network to determine why some data was weighted higher than another, has been the challenge in explaining the results we see.

There is also a concern from the European Union General Data Protection Regulation (GDPR) where a human has the right to obtain meaningful information about the logic involved, commonly interpreted as the right to an explanation.

We want to look at explainability through two factors: a local point of view and a global point of view. The global objective is to extract a general summary that is representative of some specific data set. So we explain the whole model and not just local decisions. The local objective is to explain a simple prediction as a single individual observation in the data. But you have a decision according to a neural network or a classifier or a regression algorithm, so the objective is to explain just a single observation.

There are five problems that present themselves in explainability: Instability, Transparency, Adversarial Attacks, Privacy, and Analyst Perspective.

For Instability, we look at heat maps as they are very sensitive to hyperparameters, meaning the way that we tuned that network. How we adjusted the sensitivity then impacts the interpretation. Transparency becomes more difficult the more accurate machine learning is. We call that transparency because machine learning models, neural networks, are black boxes with very high dimensionality. But what's interesting is that we can say that their prediction accuracy makes explainability inversely proportional to that. An Adversarial Attacks example is to imagine that interpretability might enable people, or programs to manipulate the system. So if one knows that for instance, having three credit cards can increase his chance of getting a loan then they can game the system by increasing their chance of getting the loan without really increasing the probability of repaying the loan. Privacy can impact your access to the original data especially in complex systems where boundaries can exist between other companies. You might not have the ability to access original data. Lastly, the Analyst Perspective. When a human gets involved to explain the system, important questions include, where to start first and how ensuring the interpretation aligns with how the model actually behaved. There are some systems by which the ML has multi-use and the human is trying to understand the perspective of use for the result given. These are some specific ways we have found that create the complexity and challenges in explainability with machine learning models.

We continue to learn and adjust based on those learnings. This is a very interesting and important topic that we will continue to explore.

Citations

Dr. Charles Forgy (1979), On The Efficient Implementation of Production Systems, Carnegie Mellon University, ProQuest Dissertations Publishing, 1979, 7919143

Nadia Burkart, Marco F. Huber (2020) A Survey on the Explainability of Supervised Machine Learning, arXiv:2011.07876 (cs)

Twitter Mentions

@kerrispa