Baidu Open Sources ERNIE 4.5: LLM Series Scaling from 0.3B to 424B Parameters

Baidu has opened the latest series of Ernie 4.5, a powerful family of basic models designed to understand enhanced language, thinking and obstetrics. The version includes ten variables ranging from 0.3B dense models to a mixture of huge experts (MEE), with the largest variable of 424B. These models are now free to the research community and the global developer through embrace of the face, allowing open experiences and broader access to Chinese and multi -language technology.

Technical overview of Arnie 4.5 architecture

The Ernie 4.5 series is based on the previous Baidu repetitions of Ernie models by introducing an advanced model structure, including dense and active MEE designs. MEE variables are particularly noticeable for the number of scaling parameters efficiently: stimulating ERNIE 4.5-ME-3B and Ernie 4.5-MOE-47B variables of experts for each input code (usually 2 of 64 experts)), while maintaining the number of active parameters while keeping the model expressive capabilities.

Ernie 4.5 models are trained using a mixture of supervision control (SFT), reinforcement learning with human comments (RLHF), and contrast technologies. The training group extends 5.6 trillion symbols across the various fields in both Chinese and English, using a multi -stage pre -training pipeline in Bidu. The resulting models show high accuracy in tracking instructions, multiple conversation, long -form generation, and thinking standards.

Typical variables and open source version

The Ernie 4.5 version includes the following ten variables:

• Dense models: Ernie 4.5-0.3B, 0.5B, 1.8B, and 4B
• Moe Models: Ernie 4.5-MOE-3B, 4B, 6B, 15B, 47B, and 424b total parameters (with varying active parameters)

For example, the ME-47B variable is active only 3B parameters while inference with a total of 47b. Likewise, the 424B model – the largest ever released by Baidu – is subject to explosive operation to make the reasoning possible and developing. These models support both FP16 and INT8 quantities for effective publication.

Performance standards

Ernie 4.5 models show significant improvements to many Chinese and multi -language NLP tasks. According to the official technical report:

• on CMMLUErnie exceeds 4.5 previous Ernie versions and achieves modern accuracy in understanding the Chinese language.
• on mmluMulti-Language Standard, Ernie 4.5-47B shows competitive performance with other leading LLMS like GPT-4 and CLADE.
• to Long -shaped generationErnie 4.5 shall achieve higher degrees of cohesion and reality when evaluated using Baidu’s internal measures.

In the tasks of tracking the instructions, models benefit from refining the contradictory contrast, which indicates an improved improved with the user’s intention and reduce hallucinations compared to previous Ernie versions.

Applications and publishing

Ernie 4.5 models have been improved for a wide range of applications:

• Chatbots and assistants: Multi -language support and the alignment of instructions makes it suitable for artificial intelligence aides.
• Answer to search and question: High retrieval and gynecology allows integration with rag pipelines.
• Content generation: The generation of rich content with long knowledge is improved with better realistic foundations.
• Code and multimedia extension: Although the current version focuses on the text, Baidu indicates that Ernie 4.5 is compatible with multimedia extensions.

With the support of the context of up to 128 thousand in some variables, the ERNIE 4.5 family can be used in tasks that require memory and thinking through the long documents or sessions.

conclusion

The Ernie 4.5 series is an important step in the development of an open source AI, providing a multi -use collection of models designed for developed and multi -language tasks and instructions. BAIDU’s decision to issue models ranging from lightweight variables to the MEE 424B lexical model is to commit to comprehensive and transparent artificial intelligence research. Through comprehensive documents, open available on face embrace, and effective publishing support, Ernie 4.5 is placed to accelerate global developments in understanding and generating natural language.

verify Paper and models embracing. All the credit for this research goes to researchers in this project. Also, do not hesitate to follow us twitter And do not forget to join 100K+ ML Subreddit And subscribe to Our newsletter.

Asif Razzaq is the CEO of Marktechpost Media Inc .. As a pioneer and vision engineer, ASIF is committed to harnessing the potential of artificial intelligence for social goodness. His last endeavor is to launch the artificial intelligence platform, Marktechpost, which highlights its in -depth coverage of machine learning and deep learning news, which is technically sound and can be easily understood by a wide audience. The platform is proud of more than 2 million monthly views, which shows its popularity among the masses.