PoE-World + Planner Outperforms Reinforcement Learning RL Baselines in Montezuma’s Revenge with Minimal Demonstration Data

The importance of symbolic thinking in the modeling of the world

Understanding how the world works is the key to creating artificial intelligence agents who can adapt to complex situations. While the models based on the nerve network, such as Draamer, provide flexibility, they require huge amounts of data to learn effectively, much more than humans usually do. On the other hand, modern methods use the synthesis of the program with large language models to create international code -based models. These are more efficient in data and can be well circulated than limited inputs. However, its use is often limited to simple fields, such as text or network worlds, where expansion of complex and dynamic environments is still a challenge due to the difficulty of generating large and comprehensive programs.

Restrictions on current programmatic models

Recent research has searched the use of programs to represent global models, and often benefits from large language models to synthesize Bithon transfer functions. Methods such as WorldCoder and CodeworldModels generate one large program, which limits the ability to expand in complex environments and their ability to deal with uncertainty and partial observation. Some studies focus on high -level symbolic models of automatic planning by combining visual inputs with abstract thinking. Previous efforts have used the bound languages ​​of the field specifically designed with specific criteria or concept related structures, such as graphs of factors in the planning networks. Theoretical models, such as AIXI, are also exploring modeling the world using TURING machines and history -based representation.

Entering the models of the world: the world’s standard and probability models

Researchers from Cornell, Cambridge, Alan Torring and Delosi Buri University offer an approach to learning the world’s symbolic models by combining many LLM small programs, each of which picks up a specific base of the environment. Instead of creating one big program, Poe-World builds a standard and probable structure that you can learn from brief demonstrations. This setting supports the generalization on new situations, allowing agents to effectively plan, even in complex games like Pong and Montezuma’s Revenge. Although it does not design raw pixel data, he learns from the observations of the symbolic object and confirms accurate modeling of exploration to make effective decisions.

Architecture

Poe-World Models as the Environmental PyTHON software called programming experts, each responsible for a specific rule or behavior. These experts are weighting and gathering to predict future countries based on previous observations and procedures. By dealing with features as an independent and conditional and learning from full date, the model remains normative and developing. Difficult restrictions improve predictions, experts are updated or trimming experts when collecting new data. The model supports planning and learning reinforcement by simulating possible future results, allowing effective decisions. Programs are manufactured using LLMS and probably interpreted, with improving expert weights by descent.

Experimental evaluation of Atari games

The study evaluates its agent, POE-World + Planner, on Atari’s Pong and Montezuma, including difficult modified versions of these games. Using the minimum explanatory show data, its method excels on basic lines such as PPO, React and WorldCoder, especially in low data settings. Poe-World explains a strong circular by precisely modeling game dynamics, even in changing environments without new offers. It is also the only way to constantly record the revenge of Montezuma. Pre-Training Policies in the Poe-World Simulation Simulation Uppervision of Learning in the Real World. Unlike the unlikely and sometimes uninterrupted WorldCoder models, Poe-World produces more detailed and knowledgeable representations, leading to better planning and more realistic behavior in the game.

Conclusion: Symbolic programs and a standard planning standard

In conclusion, understanding how the world works is very important to build adaptive artificial intelligence agents; However, traditional deep learning models require large and struggling data sets in order to update with limited input. Inspired by how human assembly and symbolic systems, the study suggests the world of Bo. This method uses large language models to synthesize program “experts” representing different parts of the world. These experts combine a composition to form an explanatory symbolic global model that supports a strong circular from the minimum data. This approach was tested on Atari games such as Pong and Montezuma, and this approach shows effective planning and performance, even in unfamiliar scenarios. Code and illustrations are available to the public.

verify Paper, project page and Jaytap page. All the credit for this research goes to researchers in this project. Also, do not hesitate to follow us twitter And do not forget to join 100K+ ML Subreddit And subscribe to Our newsletter.