How Yelp reviewed competing LLMs for correctness, relevance and tone to develop its user-friendly AI assistant

The Yelp Review app provided useful information for the debts and other consumers for decades. I have tried machine learning since its early years. During the latter explosion in artificial intelligence technology, he was still facing stumbling blocks because they worked to employ modern large language models to operate some features.

Yelp realizes that customers, especially those who only used the app, have faced a problem in communicating with the features of artificial intelligence, such as her assistant for Amnesty International.

“One of the clear lessons that we saw is that it is very easy to build something that looks great, but it is very difficult to build something that looks great and very useful,” Craig Saldan, Senior Product employee at Yelp, told Venturebeat in an interview.

Certainly not everything was easy. After I launched an assistant at Yelp, the Assistant Search Assistant for the New Akris service, in April 2024 to a broader group of customers, Yelp witnessed numbers use of artificial intelligence tools already for decline.

“The person who surprised us is when we launched this as consumers of consumers – a few users and people who are familiar with the application – [and they] I love it. We got this strong indication that this would be successful, then we put it on everyone, [and] Saldan said. “It took us a long time to find out the reason.”

It turns out that the most cross -cross Yelp users, those who visited the site or the application sometimes to find a new tailor or plumber, did not expect to speak immediately with the representative of Amnesty International.

AI features are simple to more famous

Most people are known as Yelp as a web site and an application to search for restaurant reviews and list pictures. I use Yelp to find food pictures in new restaurants and know if others share my feelings about a special nice dish. It is also a place that tells me if the café has planned to use it as a work space for this day that contains WiFi, Slops, Seating and Rarity in Manhattan.

Saldan recalled that Yolip was investing in Amnesty International for “the best part of the contract.”

“On the way back when I say in the 2013-2014 timetable, we were in a completely different generation of artificial intelligence, so our focus was to build our own models to do things like understanding the query. He said that part of the meaning of a meaningful communication function is to help people improve their research intention.”

But as AI continues to develop, as well as Yelp needs. I have invested in artificial intelligence to learn about pictures in the photos provided by users to determine the common dishes, then launched new ways to contact traders and services and help direct searches on users on the platform.

Assistant Yelp helps scream users find the appropriate “professional” to work with. People can click chat box or use claims or write the task they need. The assistant then requests follow -up questions to narrow potential service providers before formulating a message to positives that you may want to give the bid to the job.

Saldanha said that the positives are encouraged to respond to the users themselves, although he admits that large brands often have communication centers dealing with messages created by AI’s assistant in Yelp.

In addition to the Yelp Assistant, Yelp launched review and most prominent visions. LLMS user and reference feelings analysis, which combines Yelp in the degrees of feelings. Yelp uses a GPT-4O router in detail to create a set of data for a list of topics. Next, it was set using the GPT-4O-MINI model.

The most prominent review feature, which displays information from reviews, also uses a LLM router to create a data collection. However, it depends on GPT-4, with polishing from the GPT-3.5 Turbo. Yelp said it will update the feature using GPT-4O and O1.

Yelp joined many other companies that use LLMS to improve the benefit of reviews by adding better search functions based on customer comments. For example, Amazon Ruffus, an artificial intelligence assistant that helps people find the recommended elements.

Large models and performance needs

For many new AI features, including artificial intelligence assistant, Yelp turned into GPT-4O from Openai, but Saldanha noticed that regardless of the model, Yelp data is the secret sauce of its aides. Yelp did not want to lock itself in one style and keep an open mind about LLMS that will provide the best service to its customers.

“We use models from Openaii, Anthropor and other models on AWS rocks,” said Saldan.

Saldanha explained that Yelp created a model for testing models in a state of health, importance, awareness, safety of customers and compliance. He said, “It is really the best end models,” which was better. The company runs a small pilot with each model before taking into account the cost of repetition and response time.

Teaching users

Yelp is also began in an invisible effort to educate informal users and energy to be relieved with new artificial intelligence features. Saldanha said that one of the first things they realized, especially with the aides of artificial intelligence, is that the tone should have felt a person. It is not possible to respond very quickly or slowly; It can not be extremely encouraging or very bronze.

“We made a set of effort to help people feel comfortable, especially with this first response. We took nearly four months to get this second piece correctly. Once we did it, it was very clear and you can see that the hockey game is rising in the participation.”

Part of this process involves a Yelp assistant training to use certain words and a positive course. After all, Saldanha said they finally see higher numbers for AI’s features in Yelp.