[2411.01639] Know Where You’re Uncertain When Planning with Multimodal Foundation Models: A Formal Framework

View a PDF file from the paper with a title you know where you are not sure when planning with multimedia basic models: an official frame, written by Neil Bambat and 5 other authors

PDF HTML (experimental) view

a summary:Multilateral basic models provide a promising framework for automatic perception and planning by addressing sensory inputs to generate implementable plans. However, addressing uncertainty in both perception (sensory interpretation) and decision -making (the generation of the plan) is still an important challenge to ensure the reliability of the important. We offer a comprehensive framework for separating, measuring and relieving these two forms of uncertainty. We first provide a framework for uncertainty, and isolated uncertainty in the perception arising from the restrictions in visual understanding and uncertainty in the decision related to the durability of the plans created.

To measure each type of uncertainty, we suggest ways designed for the unique characteristics of perception and decision -making: we use the prediction that corresponds to calibration of uncertainty in perception and provides the official prediction that (FMDP) is to measure the uncertainty in the decision, and to benefit from the official verification techniques of the dirty guarantee. Based on this quantitative measurement, we carry out two mechanisms for the targeted intervention: the active sensing process that re -using high -resolution scenes dynamically to enhance the quality of visual entry and automatic refinement procedures that fill the form on high -certain data, and improve their ability to specifications. Experimental verification in the tasks of the real world and simulation shows that our uncertainty reduces the contrast by up to 40 % and enhances the success rates of the task by 5 % compared to the basic lines. These improvements are due to the common influence of both interventions and highlighting the importance of uncertainty, which facilitates the targeted interventions that enhance the durability and reliability of independent systems. The models, symbols and data groups that are set in this URL https are available.