ChatGPT vs. Microsoft Bing’s new A.I. chatbot: First look

Microsoft has given a small group of people early access to the new version of its Bing search engine boosted with artificial intelligence courtesy of startup OpenAI, the maker of ChatGPT.

CNBC has spent some time testing it. The new Bing can at times be more helpful, or at least more entertaining, than the usual blue links in search results. And it’s similar to ChatGPT in that it provides a lot more information than you might expect from a traditional search.

If Microsoft manages to get more people to use Bing, it could make the company even more profitable than it already is. For every percentage point that Microsoft gains in search advertising, it will pick up $2 billion in new revenue, Phil Ockenden, finance chief for the company’s Windows, devices and search divisions, said on a Tuesday conference call with analysts. “This is the largest software category that exists, and it’s incredibly profitable, incredibly large and still growing,” Amy Hood, Microsoft’s chief financial officer, said on the call.

So far, the new Bing feels like it’s been supercharged, and at the very least, people might want to try it out to see if it satisfies them more than traditional search engines that billions of people have come to know in the past 25 years.

Here’s what it’s like.

After you search on Bing, you can challenge the results rather than clicking on a few URLs or typing out a new query. To compare, I asked the current version of Bing to identify the largest software category, to which it said the answer is “enterprise software” with a citation to Statista. The new version provides similar information at the top of the search results page, but below that, you’ll find a text box in which you can type a message and kick off a chat. You might ask, “Really?” And Bing will respond with more information attempting to validate its previous answer.

That gets into the question of accuracy. You might ask the AI-boosted search engine if the response is wrong, for example. And the new chat feature will say that “one could argue that search advertising is the largest software category in the world by revenue,” and hedges by noting there are many ways to evaluate different kinds of software. That’s not what we’re used to seeing when we go to a search engine. It’s downright entertaining.

The chat feature in Bing can also perform a variety of fun maneuvers that people have come to know they can do with ChatGPT, the OpenAI chatbot that’s been available since late November. And people will surely compare it with Bing’s new chatbot.

With some queries, Bing, drawing on OpenAI’s GPT AI model, provides results that appear to go above and beyond what was asked compared with ChatGPT.

Consider the following prompt, “If I wanted to familiarize myself with the concept of German expressionism, what movies, music, and literature should I watch, listen to, and read?”‘

When that prompt is entered into ChatGPT, the OpenAI tool generates three bulleted lists detailing examples of German Expressionism in film, music and literature. The bullet points are simple and economical, containing examples of German Expressionism such as the 1920 film “The Cabinet of Dr. Caligari” and Gustav Meyrink’s novel “The Golem.”

Bing not only presents lists of cinema, music and literature representing German Expressionism, but it also gives users extra context about the artistic movement. The result looks like a Wikipedia entry about German Expressionism, complete with footnotes linking to the source material, coupled with examples of the genre that complied with the prompt’s request.

Microsoft’s supercharged Bing seems to offer better advice than ChatGPT, at least regarding the following prompt: “Create a fitness routine and meal plan for me over the next 3 months. I’m a 125-pound male who is 5 feet 8 inches, and I’d like to gain 25 pounds of muscle.”

When given that prompt, ChatGPT displays a bulleted list of a proposed fitness routine and meal plan that presumably would lead someone to gain 25 pounds of muscle in 90 days. The tips include weightlifting (45-60 minutes, four or five times a week), cardio (20-30 minutes, two to three times a week) and a dinner “that is high in protein, healthy fats, and complex carbohydrates. Examples include salmon with quinoa and vegetables or a turkey burger with sweet potato fries.”

Bing, however, notes that it might not be realistic to gain 25 pounds in three months, and it warns that doing so could be “potentially unhealthy.” Gaining that much muscle mass could “require a lot of genetic potential, steroids, or both,” Bing noted, linking out to an article from the Healthline website about the subject.

Recognizing that the search query contains a potentially harmful premise, Bing suggests that you “adjust your expectations and aim for a more reasonable and sustainable goal, such as 10-15 pounds of muscle in 3 months.”

The search tool then shares a list of some general tips to help people bulk up, including adding more protein to one’s diet, lifting weights and getting rest.

At times the Bing chatbot opted not to do things that ChatGPT would do. Bing demurred when asked to compose an email to employees telling them that some would be laid off, while others would be given recognition for their excellent performance. But OpenAI gladly created an email, subject line and all.

Bing said that coming up with such a message is “a sensitive and personal matter that requires human judgment and empathy.”

Both Microsoft and OpenAI have made it clear that their chatbots can sometimes provide inaccurate information, and CNBC encountered that on both. Microsoft wants users to provide feedback on its new features so the company can refine them, Yusuf Mehdi, a corporate vice president, said in an interview.

And when you don’t receive the sort of response you have in mind after chatting with Bing for a while, a button with a broom icon near the bottom of the window might be able to help. Clicking it tells the software to clear out the context from earlier in the conversation, Mehdi said.

Microsoft hasn’t opened up the enhanced Bing to everyone yet. People can sign up by joining a waiting list, although it currently requires people to install the Edge browser for MacOS or Windows.

WATCH: Existing jobs will be more productive because of AI, says Microsoft CEO Satya Nadella