Notes

Jordan, Michael I., and David E. Rumelhart. "Forward models: Supervised learning with a distal teacher." Backpropagation. Psychology Press, 2013. 189-236.

jordanSupervisedLearningDistal

Humans can learn movements by experiencing the distal consequences of their actions on the environment. On the contrary, supervised machine learning relies on an agent (teacher) that provides target outcomes. This assumption poses limitations in domains where supervised information is not easily accessible. How can supervised learning be employed when an unknown dynamical system transforms actions before comparing them to desired outcomes? We show that a learner’s internal model of the environment weakens the classical notion of the ”teacher”, extending the supervised learning paradigm to a larger class of problems. For example, how does a basketball player master motor commands to shoot baskets? A player wishes to score (intention) by controlling his muscles in propelling the ball (action, proximal variable). The environment dynamics determine the outcome of the shot (distal variable). Since desired outcomes are only available to distal variables, a model mapping intentions to outcomes is needed. We introduce a general approach to solving this task through a two- phase composite learning system. Firstly, the learner forms an internal forward model of the environment that maps actions to outcomes. Then, such a predic- tive model allows for solving the map of intentions to actions. Our algorithm is robust to inaccuracies in the forward model and shows several advantages over learning an inverse model directly. These properties are demonstrated in static environments and extended to dynamic ones. Our work introduces a new interpretation of the teacher’s role in the supervised learning paradigm, which expands the range of problems to which the paradigm can be applied.