Date Approved
6-19-2024
Embargo Period
6-19-2024
Document Type
Thesis
Degree Name
Master of Science (M.S.) in Computer Science
Department
Computer Science
College
College of Science & Mathematics
Advisor
Shen-Shyang Ho, Ph.D.
Committee Member 1
Jennifer Kay, Ph.D.
Committee Member 2
Nancy Tinkham, Ph.D.
Keywords
Explainable Artificial Intelligence; Machine Learning; Reinforcement Learning; Robotics; Shapley Value
Subject(s)
Artificial intelligence
Disciplines
Artificial Intelligence and Robotics | Computer Sciences
Abstract
As deep reinforcement learning (RL) models gain traction across more industries, there is a growing need for reliable agent-explanation techniques to understand these models. Researchers have developed explainable artificial intelligence (XAI) methods to help understand these 'black boxes'. While these models have been tested on many supervised learning tasks, there is a lack of examination of how these well these methods can explain hard reinforcement learning problems like robotic control. The sequential nature of learning RL policies and testing episodes create fundamentally different policies over time compared to more traditional supervised learning models. In this thesis, two important questions are explored: (1) How well do modern Shapley value based explanation techniques help understand the rationale behind actions made by robotic RL actors? (2) Can these explanations help demystify the RL training loop by estimating the predictive weight of different features throughout training epochs? Through extensive empirical experiments on multiple robotic environments and output analysis, we obtain the following observations: 1. Modern Shapley value explanation techniques often require a significant amount of extra data analysis to comprehend the logic behind trained policies. 2. Important, predictive features are found and ranked highly very early in mid-training epochs before the optimal policy is found.
Recommended Citation
Campana, Brian J., "Reinforcement Learning for Robotic Tasks: Analyzing and Understanding the Learning Process Using Explainable Artificial Intelligence Methods" (2024). Theses and Dissertations. 3250.
https://rdw.rowan.edu/etd/3250