[2505.03054] BLAB: Brutally Long Audio Bench

View a PDF file from the paper entitled Blab: Long Long Brutal Audio Seat, by OrevaOGHENE AHIA and 15 other authors

PDF HTML (experimental) view

a summary:The development of large audio language models (LMS) is able to understand the various spoken reactions is necessary to accommodate the multimedia nature of human communication and can increase access to language technologies through different user groups. The last work on LMS assessed its performance primarily in the short sound sectors, usually less than 30 seconds, with a limited exploration of the long conversation sectors that reflect the most closely user’s natural reactions with these models. We offer a long brutal audio seat (Blab), which is a long -standing audio standard that holds the audio LMS on Emiratization, estimating duration, emotion, and counting tasks using a 51 -minute sound slices. BLAB consists of more than 833 hours of various audio clips, associated with the questions and answers of the natural language concerned with humans. Our audio data was collected from a deservingly licensed sources and underwent a filtering process with the help of man to ensure compliance with the task. We evaluate six open source LMS and ownership on BLAB and find that all of them, including advanced models such as Gemini 2.0 Pro and GPT-4O, are facing tasks in Blab. Our comprehensive analysis reveals basic visions of differentials between the difficulty of the task and the duration of sound. In general, we find that the LMS sound struggles with long -shape speech, with a decrease in performance with increased duration. They perform a bad performance in resettlement, time logic, counting, and struggle to understand non -technical information, and rely on claims more than audio content. Blab acts as a difficult evaluation framework for the development of the audio LMS with the possibilities of a strong, loud -shaped sound.