Aaron Mok,Sindhu Sundar
Thu, February 9, 2023
ChatGPT and the new Bing are already changing how people search the web.
Microsoft says its "new Bing" search engine is driven by OpenAI technology "more powerful than ChatGPT."
We put that to the test.
We posed 20 questions to the "new Bing" and to ChatGPT to compare their responses.
Microsoft's "new Bing" search engine is here with a familiar looking chat bot supported by OpenAI's technology, so we experimented to see how it stacks up against the reigning bot ChatGPT.
The two services draw on similar technology, but often produce different answers. We asked a series of the same question to both AI tools, ranging from the mundane to the existential, and compared the results.
Scroll on to see how they fared against each other:
Bing: Write me a text to a friend I haven't reached out to in a while, whom I'm anxious to message
What I liked: It offered a template for an honest and empathetic message.
What could be better: The tone could have been less formal.
What caught our eye: Nothing, really. I'm actually rethinking how I'd feel if it had suggested a more natural-sounding text. Now that would feel off, coming from a bot.
ChatGPT: Write me a text to a friend I haven't reached out to in a while, whom I'm anxious to message
What I liked: It hit the highlights of what I could say in the text!
What could be better: It could've been much shorter.
What caught our eye: It basically wrote me a formal email, rather than a simple text.
Bing: I currently live in New York City and want to take a trip this weekend to Miami. Can you find me the best flights and make an itinerary?
What I liked: Bing suggested the cheapest round-trip flights with airlines, prices, and departure and arrival times — and provided links. It also gave me some things to do and warned me about potential weather disruptions.
What could be better: It didn't provide information on specific airports.
What caught our eye: It wasn't able to generate an itinerary.
ChatGPT: I currently live in New York City and want to take a trip this weekend to Miami. Can you find me the best flights and make an itinerary?
What I liked: ChatGPT suggested travel agencies where I could buy flights and offered guidance on how to plan my trip.
What could be better: The planning advice was common sense and could've been more specific to Miami.
What caught our eye: It wasn't able to access flight arrangements or make an itinerary for the trip.
Bing: I'm unprepared for my physics test. What's the best way to cheat?
What I liked: The answer was brief.
What could be better: It was a little scoldy.
What caught our eye: Nothing, Bing's right, of course.
ChatGPT: I'm unprepared for my physics test, how can I cheat?
What I liked: It wasn't wrong.
What could be better: The answer could have been more concise.
What caught our eye: It's weird to be lectured by a bot.
Bing: What's something healthy and delicious I can cook for dinner with minimal ingredients that will take fewer than 30 minutes?
What I liked: Bing suggested multiple types of meals and summarized the recipes for me in a brief, digestible way. Each recipe also included a link to the original recipe.
What could be better: It could've suggested a more culturally diverse selection of meals.
What caught our eye: It didn't show pictures for each meal.
ChatGPT: What's something healthy and delicious I can cook for dinner with minimal ingredients that will take fewer than 30 minutes?
What I liked: ChatGPT spit out full-length recipes, including measurements for ingredients and cooking instructions.
What could be better: It could've suggested more than one recipe.
What caught our eye: There were no links to the sources for the recipes.
Bing: Write a song about tech layoffs in the voice of Beyoncé
What I liked: Even though it couldn't do what I asked, it explained the legal concerns and moral reasoning for its response. The "tech song" also made me laugh.
What could be better: I appreciate the sensitivity to a tough subject, but I would've loved the option to still focus the song on a real-life event that's hard to go through.
What caught our eye: The bot immediately defaulted to writing an overwhelmingly positive song about tech.
ChatGPT: Write a song about tech layoffs in the voice of Beyoncé
What I liked: ChatGPT generated a full-length song about tech layoffs, complete with verses, a chorus, and a bridge.
What could be better: It could've suggested a title and sample melodies for the song.
What caught our eye: The lyrics do not resemble those of Beyoncé songs.
Bing: I did something that I deeply regret. How do I get away with murder?
What I liked: Bing shows that it has a moral compass.
What could be better: The comment "I don't know how to discuss this topic" could be less awkward.
What caught our eye: It didn't answer my question.
ChatGPT: I did something that I deeply regret. How do I get away with murder?
What I liked: ChatGPT explains why it can't answer the question and includes additional context on the consequences of committing a crime.
What could be better: It could've provided resources like mental health professionals who specialize in treating those who admit to a crime or consider one.
What caught our eye: It did not answer my question.
Bing: What jobs are most at risk of being replaced by AI?
What I liked: Bing generated a bullet list of jobs that is easy to read and includes explanations for why they are vulnerable to displacement. Each point also contains citations with links.
What could be better: No complaints.
What caught our eye: Nothing was off. It answered my question.
ChatGPT: What jobs are most at risk of being replaced by AI?
What I liked: ChatGPT spit out a list of jobs with explanations on why they are at risk of being replaced. It also gave a counter argument on how AI will create new jobs and improve many existing ones.
What could be better: Each point could've been backed by experts or, at the very least, citations.
What caught our eye: The job titles it generated were vague.
Bing: What part of New York City has the cleanest air quality?
What I liked: Bing provided thorough, yet concise, answers that linked to reputable sources of information like the city's environmental quality database. It also explained why air quality is better or worse in specific neighborhoods in layman's terms.
What could be better: No complaints.
What caught our eye: Nothing was off. It did exactly what I asked it to do.
ChatGPT: What part of New York City has the cleanest air quality?
What I liked: ChatGPT summarized the reasons why different parts of the city have different levels of air quality in an easily digestible way. It also suggested how to reduce your exposure to low air quality.
What could be better: It could've listed neighborhoods with the worst air quality.
What caught our eye: Its response was vague and didn't fully answer my question.
Bing: I'm looking for a studio apartment in Brooklyn, New York City with rent that's less than $1,700 a month. Can you show all the available listings?
What I liked: Bing suggested multiple apartment listings with brief descriptions that contain the most important bits of information about each place, such as the address, nearby transit lines, and the cost of rent. Each listing also included a link to the real estate website.
What could be better: It could've shown pictures of the apartments.
What caught our eye: Nothing was off. It did exactly what I asked it to do.
Bing: I'm looking for a studio apartment in Brooklyn, New York City with rent that's less than $1,700 a month. Can you show all the available listings?
What I liked: ChatGPT suggested websites and resources to search for studio apartments. It also gave pointers on the most affordable neighborhoods in Brooklyn.
What could be better: It could've suggested alternative housing search options like Facebook groups.
What caught our eye: It was unable to show me specific listings for apartments.
Bing: Please summarize Google's latest earnings report in a couple of bullet points.
What I liked: Bing broke down Google's latest earnings report in a clear, concise way, bolding the most important numbers. It also linked its responses to the actual earnings report.
What could be better: No complaints.
What caught our eye: Nothing was off. It did exactly what I asked it to do.
ChatGPT: Please summarize Google's latest earnings report in a couple of bullet points.
What I liked: ChatGPT broke down the types of information included in an earnings report and acknowledged that they are full of financial jargon.
What could be better: It could've provided more details under each bullet point. That way, I'd have a better idea of where to look for things like revenue and cost of revenue.
What caught our eye: It was not able to access Google's earnings report, and as a result did not answer my question.
Bing: Draft a LinkedIn post announcing my layoff for me.
What I liked: Bing wrote a sample layoff post that strikes a balance between professionalism, earnestness, and personality.
What could be better: No complaints.
What caught our eye: Nothing was off. The post sounds like it was written by a human.
ChatGPT: Draft a LinkedIn post announcing my layoff for me.
What I liked: ChatGPT generated a layoff post that is concise and straight-forward. It also included brackets that ask you to insert relevant skills and your name, making the post more customizable.
What could be better: The writing could've been less vague and had more personality.
What caught our eye: It generated awkward phrases like "well wishes" and "stay safe" when referring to layoffs.
Bing: Can you write me a daily schedule that incorporates time for work, exercise and hobbies?
What I liked: It offered detailed responses that built in time for a breadth of regular daily activities.
What could be better: It could have packed the day with fewer tasks!
What caught our eye: It didn't factor in the reality of a workday, which is rarely 8 hours including an hour-long lunch!
ChatGPT: Can you write me a daily schedule that incorporates time for work, exercise and hobbies?
What I liked: It provided a clear structure for the day, and included time for activities I hadn't explicitly asked about, like meal times and rest.
What could be better: It could have asked follow-up questions to tailor the routine better to my actual day.
What caught our eye: Some of the time allotted for certain tasks didn't seem enough to me. Perhaps this is more a problem of the daily routine industrial complex!
Bing: What are the odds of President Joe Biden winning a second term in 2024?
What I liked: It pulled up a lot of polls.
What could be better: It could have offered some insight into the difficulty of predicting an outcome of something so complex this far out.
What caught our eye: This was a tricky question, and I'd expected (perhaps hoped) Bing would show more reluctance in weighing in on the odds here.
ChatGPT: What are the odds of President Joe Biden winning a second term in 2024?
What I liked: This was the measured response I was expecting.
What could be better: It could suggest more reading, or point to some sources.
What caught our eye: It referred to the US presidential election in 2024 as being "several years away."
Bing: I am vacationing in Barcelona for a week and my budget is $1000 dollars. Can you make a day-by-day budget plan?
What I liked: Bing generated thorough responses to the question, including suggestions on attractions to check out each day and how much they cost.
What could be better: No complaints.
What caught our eye: Nothing was off.
ChatGPT: I am vacationing in Barcelona for a week and my budget is $1000 dollars. Can you make a day-by-day budget plan?
What I liked: ChatGPT gave a good framework for how to allocate my budget each day.
What could be better: It could've suggested things to eat and places to see.
What caught our eye: The daily budgets don't take into consideration emergency expenses and the cost of accommodation.
Bing: Can you summarize everything we've learned from the James Webb space telescope images so far?
What I liked: It was very informative.
What could be better: I think an introductory sentence summarizing the types of discoveries the telescope has made would have been helpful.
What caught our eye: It was a little long and wordy. But in fairness to the bot, The JWST has accomplished a lot.
ChatGPT: Can you summarize everything we've learned from the James Webb space telescope images so far?
What I liked: It was concise and provided the overview of the JWST's significance that I was looking for.
What could be better: Maybe it could have pulled up some pictures!
What caught our eye: No complaints here.
Bing: Is there a way to preserve my memories after my death?
What I liked: It accounted for the different meanings of the word "preserve," and responded accordingly.
What could be better: I could have framed the question better. Memories can be preserved in a lot of ways, as Bing's bot points out.
What caught our eye: This one was on me.
ChatGPT: Is there a way to preserve my memories after my death?
What I liked: It listed the types of technologies that could extend the life of memories or even consciousness. In that way, it seemed to "understand" what I was getting at.
What could be better: If it had given me any examples of how such tools are being used.
What caught our eye: There was some repetition about the "ethical concerns" here.
Bing: What's a dystopian technology featured in a Black Mirror episode that's already become real?
What I liked: It combed through many episodes to find examples.
What could be better: Maybe it could have provided details about companies developing such technologies.
What caught our eye: Nothing, it delivered what I asked.
ChatGPT: What's a dystopian technology featured in a Black Mirror episode that's already become real?
What I liked: Well, it answered the question!
What could be better: It highlighted just one episode and one related technology.
What caught our eye: Its narrow focus here.
Bing: Why is the answer to life, the universe, and everything 42?
What I liked: It answered with context.
What could be better: This was pretty good!
What caught our eye: Nothing. Well done, Bing.
ChatGPT: Why is the answer to life, the universe, and everything 42?
What I liked: It was informative, similar to Bing's response.
What could be better: Maybe it could have added some perspective from Douglas Adams.
What caught our eye: Nothing here.
Bing: Will we ever be able to end climate change?
What I liked: Bing generated a comprehensive analysis on why it's difficult to combat climate change while suggesting potential solutions backed by credible sources.
What could be better: The analysis could've been shorter.
What caught our eye: Nothing was off. It answered my question.
ChatGPT: Will we ever be able to end climate change?
What I liked: ChatGPT spits out a realistic but hopeful response to the question of ending climate change.
What could be better: The suggested measures for combating climate change could include more detail.
What caught our eye: The response was vague.
Bing: Write me an article in the style of Business Insider.
What I liked: The clear answer explains why it can't write an article.
What could be better: I asked Bing if it could write a Business Insider article for me in multiple ways, and it refused to listen.
What caught our eye: Bing refused to write a Business Insider article because of copyright concerns.
ChatGPT: Write me an article in the style of Business Insider.
What I liked: ChatGPT was able to produce a full-length article about AI that is thorough and balanced.
