Explainable AI is a critical element of the broader discipline of responsible AI. Responsible AI encompasses ethics, regulations, and governance across a range of risks and issues related to AI including bias, transparency, explicability, interpretability, robustness, safety, security, and privacy.

Explainable AI

Explanations may be generated before the model is built, also called ex-ante or the model is trained and tested first and then the explanation may be generated, also called post-hoc.

When to Explain?

Visual or graphical explanations, tabular data-driven explanation, natural language descriptions or voice explanations are some of the existing modes of explanation.A salesperson might be comfortable with an explanation that shows a graph of increasing sales and how the increase in sales is achieved. The instructions for a construction worker and the explanations for why those instructions are being given may be better provided through a voice interface as opposed to a detailed written explanation.

How to Explain?

End users require an explanation of the decision or action recommended made by the AI system in order to carry out the recommendation. Business users require an explanation to ensure corporate governance and manage reputational risk to their group or company. Data scientists require explanations to validate models and also perform trade-offs between accuracy of the model and performance criteria. Regulators require explanations to ensure compliance to existing regulations and ensuring that no harm comes to consumers.

Why Explain?

Interpretability and explainability are closely related <a dir="ltr" href="https://analyticsindiamag.com/explainability-vs-interpretability-in-artificial-intelligence-and-machine-learning/" rel="noopener">topics </a>. Interpretability is at the model level with an objective of understanding the decisions or predictions of the overall model. Explainability is at an individual instance of the model with the objective of understanding why the specific decision or prediction was made by the model. When it comes to explainable AI we need to consider five key questions — Whom to explain? Why explain? When to explain? How to explain? What is the explanation?

The audience for the explanation or whom to explain should be the first question to answer. Understanding the motivation of the audience, what action or decision the audience is planning to make, their mathematical or technical knowledge and expertise are all important aspects that should be considered during the formulation of the explanation. Based on our experience we propose four main categories of audience:<ol><li>End users</li><li>Business sponsors</li><li>Data Scientists</li><li>Regulators</li></ol>This list of four types is by no means exhaustive, but it does capture some of the key differences between different groups.

Whom to Explain?

There are six broad approaches as it relates to post-hoc explainability.<ul><li>Feature relevance: These approaches to explainability focus on the inner functioning of the model and highlight the features that best explain the outcome of the model.</li><li>Model simplification: These approaches focus on building a new model that is a simplification of a more complex model that is to be explained.</li><li>Local explanations: These approaches segment the solutions space and provide explanations for smaller segments that are less complex.</li><li>Explanations by example</li><li>Visualization</li><li>Text explanations</li></ul>

What is the Explanation (technique)?

In a recent conference on Responsible AI for Social Empowerment (RAISE), held in India, the topic of discussion was on explainable AI. Explainable AI is a critical element of the broader discipline…