Code as PoliciesA framework utilizing language model-generated programs to enable robots to perform complex tasks through code-based policies.

Code as Policies is a framework utilizing language model-generated programs to enable robots to perform complex tasks through code-based policies. It allows robots to understand and execute natural language instructions by using large language models to write robot policy code. This framework showcases its capabilities through demonstrations of tabletop manipulation tasks, such as arranging blocks and placing them in bowls, and can be applied in a variety of domains. By chaining classic logic structures and referencing third-party libraries, the generated code can exhibit spatial-geometric reasoning, generalize to new instructions, and prescribe precise values to ambiguous descriptions. The framework employs a few-shot prompting approach to write robot policies that can represent reactive policies and waypoint-based policies. It can write more complex code and improves state-of-the-art in solving problems on the HumanEval benchmark. Code and videos demonstrating the framework's capabilities are available at its GitHub repository. In the tabletop manipulation domain, the framework uses natural language arguments to compose the generated code via function calls. Prompts are used to specialize the language model to perform different functions. The framework has demonstrated its capabilities in various tasks, including arranging blocks in a square, moving blocks to specific positions, and even executing commands that involve creative storytelling. However, the framework's reliance on large language models means that it may be limited by their capabilities and biases. Additionally, the use of natural language arguments may introduce ambiguities or uncertainties in the generated code.