[2503.13222] Can Language Models Follow Multiple Turns of Entangled Instructions?

View the PDF file from the paper entitled “Can you follow the multiple turning offs of language models from interlocking instructions?

PDF HTML (experimental) view

a summary:Despite the great achievements in improving the capabilities of the instructions for the LLMS models, the ability to address potential or conflicting instructions remain a major challenge. Real world scenarios often require consistency through multiple instructions over time, such as secret privacy, personal preferences, and priorities, which require advanced capabilities to integrate multiple turns and carefully balanced goals when instructions or conflict intersect. This work provides a systematic investigation of LLMS capabilities in dealing with multiple turns of instructions, and cover three levels of difficulty: (1) Recover information from instructions, (2) tracking and thinking via turns, (3) resolving disputes between instructions. We build Multiturninstruct ~ with high -quality multi -quality conversations of $ 1.1,000 through the human approach in the episode and lead to nine categories of ability, including statistics, dynamics, thinking, and multiplicity. Our discovery reveals an interesting comparison between different capabilities. While GPT models show highly reservation, they show a decrease in effectiveness in privacy protection tasks that require blocking selective information. Large models show stronger thinking capabilities but are still struggling to solve conflicting instructions. More importantly, these performance gaps can only be attributed to the loss of information, as models show strong degrees in memorization tasks. However, their attention mechanisms fail to integrate multiple instructions effectively. These results highlight the critical areas of improvement in the complex real world tasks that involve multiple instructions. Data and symbols are released in this URL https.