Overview

Explainable machine learning in deployment

Bhat et al. interviewed 20 data scientists to examine their explainablility techniques. After determining that 4 techniques were widely used they described how this was used most often and finally they provided some recommendations and concerns in relation to explainability.

What you should know before reading further:

  • For who: The complete ML community
  • Which phase: Building
  • Ethical principles: Explicability ; transparency
  • Type of Machine Learning: Unsupervised
  • Type of Explainability: different types of explainability are introduced.

The article is interesting as it introduces a number of explainability techniques as currently used in the field.

These methods include:

  • Feature Importance via Shapley Values
  • Counterfactual explanations
  • Adversial Training
  • Influential samples

Besides the discussion of these techniques the authors highlight important concerns that should be kept in mind when using explainability.

  1. Causality should not be the sole focus as this would exclude interesting data/events from consideration
  2. Privacy and the possible loss of privacy needs to be a consideration while using these techniques.
  3. The use of explainability can lead to better performance. The authors note that this performance can then be used in ways that the stakeholders might not necessarily agree with. Their example concerned better natural language processing in order to be able to better moderate social media posts.

And finally the authors provide the suggestion, based on their interviews, that designing explainability with and for (external) stakeholders will be an important feature if we want to take the explainability from an internal tool used by the data scientist and engineers to a tool providing explanations to external stakeholders. In order to achieve this they suggest to:

  • Identify stakeholders;
  • Investigate what explainability means for these stakeholders;
  • Determine the effect of an explanation on a stakeholder and design your explanations with this in mind.

Links

Explainable Machine Learning in Deployment

Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. 2020. Explainable machine learning in deployment. In /Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency/ (/FAT* ’20/). Association for Computing Machinery, New York, NY, USA, 648–657. DOI:https://doi.org/10.1145/3351095.3375624

https://arxiv.org/abs/1909.06342