Help
Welcome All Technologies Browse by Category

Explainable Decisions of Algorithms Using Examples

Technology #2019-023

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Image Gallery
Categories
Researchers
Patrick Shafto, PhD
Dr. Patrick Shafto is the Henry Rutgers Term Chair in Data Science and Associate Professor in the Department of Mathematics and Computer Science at Rutgers - Newark. He is also affiliated with the Institute for Data Science, Learning and Applications (I-DSLA) and has appointments in Psychology, Rutgers Business School, and the Center for Molecular and Behavioral Neuroscience (CMBN).
Managed By
Andrea Dick
Assistant Director, Licensing 848-932-4018

Artificial Intelligence algorithms (AI) and Machine Learning (ML) are used across a widening array of application domains. To trust the decisions made by these algorithms it is important for people to understand how they arrive at them.

Summary:

Researchers at Rutgers University have created a novel solution that uses Bayesian Teaching for explaining the decisions made by AI algorithms.  Bayesian Teaching is a model-agnostic system that samples data subsets to explain model inferences to a domain (but not necessarily technical) expert. 

This solution allows a user to ask questions of a ML model (used by an AI Algorithm to make a decision) and in response it provides (e.g., displays) examples to explain the reasons for the decisions.  For a use case, consider a user who wants to know why an AI algorithm in a self-driving automobile made a specific decision; our solution provides responses such as: “in similar situations it was found that the reason for this decision were: “example reason 1”, “example reason 2”, etc.  The examples might be in the form of displayed images (see above) or some other form relevant to the type of application/use.

Advantages:

  • Use of bayesian teaching ensures that the best sets of examples are generated according to a model.

  • Supports any type of machine learning model that has probabilistic interpretation (e.g., supervised, unsupervised and reinforcement learning (including deep learning)) models.   

Applications: This invention can be used in any functional area where explanations of decisions of AI/ML algorithms are desired or necessary. Examples are:

  • Enables the enforcement of the requirements of European Union General Data Protection Regulation (GDPR) compliance.

  • Provides the ability to determine whether machine learning models cause decisions that discriminate based on race and ethnicity (bias detection).  For example, to determine if there is bias in AI/ML applications that perform bank loan and mortgage processing, resume processing for matching candidates/positions, and targeted advertising,

  • Applies to almost all applications of AI/ML

Intellectual Property & Development Status:

The technology is patent pending and is currently available for licensing.