An Enhanced Task Framework for Instruction Following in Minecraft Dialogues

PDF view of the paper entitled BAP V2: a strengthening framework for the following education in Minecraft dialogues, by Prashant Jayannavar and 8 other authors

PDF HTML (experimental) view

a summary:The development of interactive agents can understand the language, depict their surroundings, and behavior within the material world is a long goal of artificial intelligence research. The task of building the cooperative Minecraft (MCBT) (NARYAN-Cen, Jayannavar, and HOCKENMAIER 2019, a game of players in which the architect (A) presented based on this goal. In this work, we focus on the sub -tasks of predicting construction actions (BAP): prediction of actions B in the context of a multimedia game (Jayannavar, Narayan -Chen, and Hockenmaier 2020) – a difficult test for land instructions, with limited training data. We completely re -examine this task and offer the BAP V2 to face the main challenges in evaluation, training and modeling data. Specifically, we define an improved standard for evaluation, and features a cleaner and more insightful and more insightful test set that also reveals spatial thinking as the bottleneck neck. To process data scarcity and teach basic spatial skills, we create different types of artificial MCBT data. We note that the current LLM -based SOTA models trained in human BAP dialogues fail on these simple and planning models, but it shows that training models on these artificial data improve their performance in all fields. We also offer a new Sota model, Llama-Crafts, which enhances richer input representations, and the F1 53.0 score on the BAP V2 mission and strong performance on artificial data. Although this result improving a prominent 6 -point improvement over the previous work, it also emphasizes the difficult difficulty of the task, establishing the BAP V2 as fertile ground for future research, providing a useful scale for spatial capabilities for current text LLMS only in such embodied tasks.