AI Linguistics: Language Models Master Metalinguistics

In a recent study, the researchers show that the OPNAI thinking model is able to identify one of the most complex phenomena in the human language, a concept called linguistic linguistic. Ladded includes a nesting element within another element in the sentence; It resembles a lake on an island in a lake. The results were published on June 3 in IEEE transactions on artificial intelligence.

Gašper Beguš is a associate professor of linguistics at the University of California, Berkeley, with deep interest in language and intelligence. His research compares human and human forms of learning to understand their differences and strengths, as well as to understand the limits of artificial intelligence from the point of view and organizational.

Can LLMS do metal scientists?

In the new study, BEGUš and its metal linguistic aides for four large linguistic models (LLMS): Openai GPT-3.5 Turbo, GPT-4 and O1, as well as Meta’s Llama 3.1. While many studies have explored the quality of these models of language production, this study specifically looked at the ability of models to analysis Language – Its ability to perform mineral individual scientists.

For example, when the sentence has multiple meanings, are language models capable of identifying and “understanding” all different meanings? Beguš provides a simple example of one word for this challenge. “Unprecedented He has two meanings, right? He explains: “Either you cannot open it, or you can open it.”

In their studies, the researchers tested artificial intelligence models with a difficult whole sentence that can have multiple meanings, called mysterious structures. For example: “Elisa wanted to take it out.”

The sentence can express Elisa’s desire to get a person out of a group, or to remove it. While all the four models of the language are properly identified as a mysterious structure, only O1 was able to set the different meanings that the sentence could correctly contain.

Llms load capabilities

Beguš confirms that the most important progress that has been reported in this study is the O1 ability to successfully engage in linguistic linguistic. An example of a lump element within the sentence appears in the parentheses in the following sentence: “The global view [that the prose Nietzsche wrote expressed] It is unprecedented. [that the prose [Nietzsche wrote] Express]unprecedented. “

In the experience of linguistic linguistic, the researchers asked the language models to determine whether a specific sentence is repeated, determining the lukear part, drawing a stereotype that represents the sentence, and adding another layer of lukear to the sentence.

All four models can determine the camel sentences, but O1 greatly outperformed the other models when it comes to properly planning the complex sentence structure, achieving a degree of 0.87 out of 1 compared to an average degree of 0.36 for the ancient artificial intelligence models.

Beguš notes that the analysis of these lubrication sentences is not an easy task. “These are the most complex types of sentences even for humans for analysis,” he says. This return adds He is A Determine the characteristic of the human language, and one Which long ago The captured linguists. There is no other animal for him This complexity showed in Communications. BEGUš says that the fact that artificial intelligence models can identify and analyze the lukear that are able to obtain a high level of linguistic complexity.

To what extent can you go llms?

The researchers also tested the ability of the models to analyze the vocal rules, which are the organization of sounds within the language. In this experience, researchers used the languages ​​that he invented so that artificial intelligence models did not depend on memorization, but instead analyzed the structure of the word itself. For example, the forms were asked to set the date for the residents pronounced for a long or short period. Once again, O1 greatly outperformed the other models, as the correct conditions of the vocal rules were identified in 19 out of 30 cases.

Beguš emphasizes the need to understand the extent that these models can reach with their linguistic abilities, especially for safety and organization purposes. He says: “We see that the goal is already high, and that they reach it.”

But he wonders how much the models can go. Can they succeed in analyzing three layers of load? What about five or 10? “Where do you do? [the models] Stop? Because the goal of this research is really an understanding, what are its limits? ” He says. “R.Catter of one million dollars.“