Agentic Query Rewriting and Evaluation for Complex Document Processing

View the PDF file from the paper entitled DOCETL: Re -writing and evaluating parents to address complex documents, by Shrea Shankar and 4 other authors

PDF HTML (experimental) view

a summary:Non -structured data analysis was a continuous challenge in data processing. LLMS models have shown a promise in this regard, which led to recent proposals for the definition frameworks for processing unsheather data. However, these frameworks focus on reducing the cost when carrying out the user -specific processes using LLMS, instead of improving accuracy, and carrying out most processes as it is (in one LLM call). This is a problem for complex tasks and data, as LLM outputs of the user’s knowledge of the user are often inaccurate, even with improved claims. For example, LLM may struggle to determine {\ em all} from the specified sentences, such as force majeure or compensation, in lengthy legal documents, which require decomposition of data, task or both.

We offer DOCETL, a system that improves complex document processing pipelines, with LLM insufficiency calculating. DOCETL provides an advertising interface to users to determine these pipelines and uses a based approach to the agent to automatically improve it, benefit from rewriting the new agent (which we call rewriting directives), as well as the improving and evaluation framework. We offer (I) Re -writing logical pipelines, which are specially designed for LLM tasks, (2) mechanism to assess the plan directed to the worker that collects and regulates the validity of the task, and (3) the improvement algorithm that finds promising plans, taking into account the estimates based on the plan based on the agent. Our evaluation of four different tasks for analyzing unsheather documents shows that DOCETL finds plans with more accurate outputs by 25 to 80 % of well -engineering baseline lines, as it deals with a critical gap in irritable data analysis. Docetl open source in this URL http, and until March 2025, collected more than 1.7 kg of Jaythb stars, as users extended a variety of fields.