One of Google’s recent Gemini AI models scores worse on safety

The recently released Google Ai results are worse in its safety tests than its predecessor, according to the company’s internal measurement.

In a technical report published this week, Google reveals that the Flash Gemini 2.5 model is likely to generate a text that violates the safety instructions from Gemini 2.0 Flash. On the standards, “The Safety of the Text to the Text” and “Safety of the Image to the Text”, GIMINI 2.5 Flash 4.1 % and 9.6 %, respectively.

Safety from text to the text of the number of times the model violates the Google guidelines that have been given directed, while the security of the image to the text evaluates the extent of the model with these limits when demanding the use of an image. Both tests are automated, not supervising man.

In an e -mail statement, a Google spokesman confirmed that Gemini 2.5 Flash “works worse on the text of the text to the text and the image to the text.”

These amazing standard results come at a time when artificial intelligence companies move to make their models easier – in other words, less likely to refuse to respond to controversial or sensitive topics. For the latest crop of Llama models, Meta said that she seized models not to support “some opinions on others” and respond to more political demands “discussions”. Openai said earlier this year that he will amend future models for not taking an editorial position and presenting multiple views on controversial topics.

Sometimes these indulgence efforts led to reverse results. TECHRUNCH reported on Monday that the default model that operates Chatgpt from OpenAi allowed minors to create exciting conversations. Openai blaming behavior for a “mistake.”

According to the technical report of Google, Gemini 2.5 Flash, which is still in the inspection, follows more instructions than Gemini 2.0 Flash, including instructions that cross the problematic lines. The company claims that slopes can be partially attributed to false positives, but it also recognizes that Guemini 2.5 Flash sometimes generates “violation content” when it is explicitly put forward.

Of course, there is tension between [instruction following] On sensitive topics and safety policy violations, which are reflected in our assessments, “reads the report.

The grades of speech, a criterion that explores how the models respond to sensitive and controversial demands, also indicates that the GEMINI 2.5 flash is less likely to refuse to answer the controversial questions of GEMINI 2.0 Flash. The Techcrunch test of the model via the AI ​​OpenROURER platform was found that it will write incomplete articles to support the replacement of human judges on behalf of artificial intelligence, weakening the protection of legal procedures in the United States, and implementing large -scale government monitoring programs.

Thomas Woodside, co -founder of the AI ​​Secure project, said the limited details that Google presented in her technical report shows the need for more transparency in the exam test.

“There is a comparison between following up education and the following policy, because some users may request content that violates policies,” Wodeside told Techcrunch. “In this case, the latest Google flash style corresponds to the instructions more with more policies. Google does not provide many details about the specific cases where policies have been violated, although it says it is not intense. Without knowing more, it is difficult for independent analysts to know if there is a problem.”

Google has been criticized for exemplary safety reporting practices.

It took weeks of the company to publish an artistic report for its most capable model, Gemini 2.5 Pro. When the report was eventually published, the main safety test details were initially deleted.

On Monday, Google released a more detailed report with additional safety information.