[2406.01698] Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models

View the PDF file from the paper entitled “Removing mystery from the AI ​​platform” for the inference distributed to the next generation models of LLM models, written by Abhimanyu Bambhaniya and 7 other authors

PDF HTML (experimental) view

a summary:LLMS models showed great performance across a wide range of applications, and often outperform human experts. However, the publication of these giant models efficiently requires the use of various inferences, carefully designed devices with computing, memory and wide network resources. Through the continuous innovation in LLM, the model improvements and architectural engineering that develops rapidly break, the requirements of devices remain to meet the goals of the service level (SLOS) is an open research issue.

To answer the question, we offer an analytical tool, Genz, to efficiently navigate the relationship between the various LLM structure (density, GQA, MEE, MAMBA), LLM that serves improvements (cutting, speculative decoder, Quanitization), and AI platform design parameters. Our tool is estimated at the LLM performance performance of the specified scenario. Its health has been validated for real devices platforms that operate different different LLM models, as they made a MAX geographic error for the URL HTTP using Genz to determine memory capacity, memory domain display, network chronic, and network frequency display requirements through various User use User use. We also study the diverse architectural options used today (inspired by LLM service platforms from many sellers) to help inform computer engineers of the design of urgent ai and devices from the next generation. The trends and visions derived from Genz can direct the artificial intelligence engineers who publish LLMS as well as computer engineers who design the accelerators and platforms of the next generation. Ultimately, this work sheds light on the platform design considerations to cancel the possibility of full capabilities of large language models through a set of applications. The source code is available in this URL https. It can also be tried on the URL https without any preparation on your web browser.