Robot Program Construction via Grounded Natural Language Semantics & Simulation

Robots acting in semi-structured, human environments need to understand the effects of their actions and the instructions given by a human user. Simulation has been considered a promising reasoning technique to help tackle both problems. In this paper, we present a system that constructs an executable robot program from a linguistic semantic specification produced by parsing a natural language sentence; in effect, our system grounds the semantic specification into the produced robot plan. The plan can then be run in a simulated environment, which allows one to infer more about the plan than was present in the initial semantic specification. Our system allows modeling how actions can be modified by subclauses, which we showcase by a transport action. Simulation runs allow discovery of better parameters, either locally for a subtask or such that the entire task is better performed; simulation reveals these parameterizations may differ.