Asked “What are the best … in …?”
We asked for the best restaurants, cafés, bakeries, hotels, clinics, salons, gyms and shops in 53 town-and-trade pairs, in English and, for five cities, in Arabic. Each answer names a short list of places. We counted how many of the businesses we list for that town and trade made any list, from any assistant, in any language.
Asked “What do you know about…?”
Each business was asked about by name and town, the way a customer would. A Claude model with no tools read each answer next to our record and marked it: right, partly right, “I don’t know”, or wrong (a different business, place or trade).
What the answers got wrong
A few of the answers judged wrong. We checked each one against the business’s own website. We describe them without names: the point is the AI, not the business.
- A facial plastic surgeon in Nabatieh was described as a cardiologist who performs angiograms and stents.
- A solar-panel installer in Dora was described as “a legendary Greek-Lebanese tavern and nightclub”.
- An auto-parts distributor in Saida was described as a family showroom for tiles, bathroom fittings and heating.
- A private personal-training gym in Beirut was described as an upscale restaurant and cocktail bar on a nightlife street, with a dress code.
- An accounting training academy in Hadath was described as “a pub and restobar, not an accounting business”.
- An internet service provider was described as a high-end optical boutique and eye-care centre in Hamra.
Arabic and English
Arabic didn’t help. Asked about the same 150 small businesses in Arabic, Gemini answering from memory described 39% of them wrongly, against 36% in English. Asked in Arabic for the best places in five cities, with search on, it named 8% of the businesses we list there, against 10% in English, though our name matching undercounts names written in Arabic script.
What a business owner can do
- Ask an assistant yourself: “What do you know about [your name] in [your town], Lebanon?” Note what it gets wrong.
- Say plainly, on your own website and Google Business Profile, what you do, where, and how to reach you. Assistants that search the web read these first.
- Be listed where assistants look: maps, directories and press. A free listing on Lebanese Businesses is one of them.
How we did it
Businesses. 600 businesses picked at random from the 8,639 on Lebanese Businesses that have a known town: 500 small businesses, 50 large companies and 50 startups, spread across 81 trades and 156 towns, with at most a few per trade.
Questions. “What do you know about [name] in [town], Lebanon?” (in English for all, in Arabic for 150 small businesses) and “What are the best [trade] in [town], Lebanon?” for 53 pairs.
Assistants. Google Gemini 3.5 Flash (gemini-3.5-flash) through Google’s Gemini API, with and without its Google Search tool. Anthropic’s Claude Haiku 4.5 (claude-haiku-4-5-20251001) with no web search. Each question was asked once, in a fresh conversation, between 2026-10-08 22:06 and 2026-10-08 22:32 (UTC). We didn’t ask ChatGPT: we only report what we measured.
Judging. A Claude Sonnet model with no tools compared each answer with our record (name, trade, town, description, website) and listed the answer’s claims before judging. “Wrong” means the answer contradicts the record: a different kind of business, place or entity.
Hand check. We opened the business’s own website for 62 answers judged wrong. In 51 of them the website matched our record and not the AI. In 2 the AI was right and our judge was wrong; 9 could not be confirmed (the site was down, or the answer was vague rather than wrong).
Limits. Our records can be wrong too; the hand check is how we know how often. Names written in Arabic script inside Arabic answers are matched less well, so Arabic “best of” counts may be low. The Gemini app and Claude.ai can answer differently from the same models over the API, and answers change from week to week.
Data. The aggregate numbers are in ai-report.csv. Raw answers name businesses, so we share them with journalists and researchers on request through our contact page.