Learning from Human Teachers: Supporting How People Want to Teach in Interactive Machine Learning
MetadataShow full item record
As the number of deployed robots grows, there will be an increasing need for humans to teach robots new skills that were not pre-programmed, without requiring these users to have any experience with programming or artificial intelligent systems. To enable this, we need to better understand how people want to teach and to support the ways in which people want to teach. To learn from human teachers, we consider the case where a human could provide online evaluative feedback, or design a sequence of tasks for the agent to learn on. With respect to learning from evaluative feedback, this dissertation demonstrates that learning algorithms that treat human feedback as a complex, discrete mode of communication can be better suited to learning from human trainers, rather than simply a numeric utility function to be optimized. Our empirical results indicate that humans, when teaching agents, deliver discrete feedback and follow different training strategies. We develop a novel model of evaluative feedback that captures knowledge about a teacher's training strategy. Based on this model, we develop two Bayesian learning algorithms that can learn from real users more efficiently than previous approaches that interpret feedback as numeric. To address limited evaluative feedback, we design a new representation of the learning agent. We demonstrate empirically that by changing the speed of the agent according to its confidence level, human trainers can be implicitly motivated to provide more explicit feedback when the learner has more uncertainty about how to act. We believe this can potentially be an effective way for the agent to interact with end-users, when taking into account human factors such as frustration. Finally, we consider the case where a human could design a sequence of tasks for the agent to learn on. We investigate how non-experts design curricula and how we can adapt machine-learning algorithms to better take advantage of this non-expert guidance. We empirically show that non-experts can design curricula that result in better overall agent performance than learning from scratch. We also demonstrate that by leveraging some principles people use when designing curricula, we can significantly improve our curriculum-learning algorithm.
Showing items related by title, author, creator and subject.
The learning environment as place : an analysis of the United States Department of Education's six design principles for learning environments Fritz, Catherine Mary, 1957- (2003)
Herman, David (2019)With the rapid increase in the amount of online learning opportunities available has come with it an increase in challenges concerning how to support effective teaching and learning in online learning spaces. Through two ...
Tripp, Lisa Marie (2011)A debate exists concerning the source of performance decrements observed in the literature during implicit learning of multimodal compared to unimodal sequences. Some authors argue that these decrements are due to erroneous ...