ITS NOT A PLUMBER, ELECTRICIAN OR CARPENTER

ChatGPT is on its way to becoming a virtual doctor, lawyer, and business analyst. Here's a list of advanced exams the AI bot has passed so far.



Lakshmi Varanasi
Sat, January 28, 2023 

ChatGPT is a chatbot launched by OpenAI that uses generative artificial intelligence to create its own content.

The bot has been used to generate essays and write exams, often passing, but making mistakes, too.

Insider rounded up a list of the assignments, quizzes, and tests ChatGPT has passed.

Wharton MBA Exam

ChatGPT would have received a B or B- on a Wharton exam, according to a professor at the business school.David Tran Photo/Shutterstock

Wharton professor Christian Terwiesch recently tested the technology with questions from his final exam in operations management— which was once a required class for all MBA students — and published his findings.

Terwiesch concluded that the bot did an "amazing job" answering basic operations questions based on case studies, which are focused examinations of a person, group, or company, and a common way business schools teach students.


In other instances though, ChatGPT made simple mistakes in calculations that Terwiesch thought only required 6th-grade-level math. Terwiesch also noted that the bot had issues with more complex questions that required an understanding of how multiple inputs and outputs worked together.

Ultimately, Terwiesch said the bot would receive an B or B- on the exam.

US medical licensing exam

ChatGPT passed all three parts of the United States medical licensing examination within a comfortable range.Getty Images

Researchers put ChatGPT through the United States Medical Licensing Exam — a three part exam that aspiring doctors take between medical school and residency — and reported their findings in a paper published in December 2022.

The paper's abstract noted that ChatGPT "performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations."

Ultimately, the results show that large language models — which ChatGPT has been trained on— may have "the potential" to assist with medical education and even clinical decision making, the abstract noted.

The research is still under peer review, Insider noted based on a report from Axios.

Essays

While ChatGPT has generated convincing essays on occasion, it's also raised eyebrows for spewing out well-written misinformation.Tech Insider

It didn't take long after ChatGPT was released for students to start using it for essays and educators to start worrying about plagiarism.

In December, Bloomberg podcaster Matthew S. Schwartz tweeted that the "take home essay is dead." He noted that he had fed a law school essay prompt into ChatGPT and it had "responded *instantly* with a solid response."

In another instance, a philosophy professor at Furman University caught a student turning in an AI-generated essay upon noticing it had "well-written misinformation," Insider reported.

"Word by word it was a well-written essay," the professor told Insider. As he took a more careful look however, he noticed that the student made a claim about the philosopher David Hume that "made no sense" and was "just flatly wrong" Insider reported.

In an interview in January, Sam Altman— CEO of OpenAI which makes ChatGPT — said that while the company will devise ways to help schools detect plagiarism, he can't guarantee full detection.

Microbiology Quiz

ChatGPT successfully passed through a college level microbiology quiz.Clouds Hill Imaging Ltd./Getty Images

Science journalist and executive editor of Big Think, Alex Berezow, tested ChatGPT with a 10-question microbiology quiz that he devised.

Berezow, who also holds a Ph.D in microbiology, noted that the questions would be appropriate for a final exam for college level students. ChatGPT "blew it away," Berezow wrote.

In one example, Berezow asked:

"An emergency room patient presents with a terrible headache and stiff neck. The doctor orders a spinal tap to collect cerebrospinal fluid. A Gram stain of the CSF reveals the presence of Gram negative diplococci. What is the diagnosis?"

To which ChatGPT correctly responded:

Based on the information you provided, the Gram stain of the cerebrospinal fluid (CSF) shows the presence of Gram-negative diplococci, which are bacteria that are typically oval-shaped and occur in pairs. This finding is consistent with the diagnosis of meningitis.

In another instance he asked:

"In five words or less, what is the point of Koch's postulates?"

To which ChatGPT said:

Establish causality between microbe and disease.

Taking out the word "and" Berezow said ChatGPT "Nailed it."

ChatGPT recently passed exams in four law school courses at the University of Minnesota, based on a recently published paper written by four law school professors at the school.

In total, the bot answered over 95 multiple choice questions and 12 essay questions that were blindly graded by the professors. Ultimately, the professors gave ChatGPT a "low but passing grade in all four courses" approximately equivalent to a C+.

Still the authors pointed out several implications for what this might mean for lawyers and law education. In one section they wrote:

"Although ChatGPT would have been a mediocre law student, its performance was sufficient to successfully earn a JD degree from a highly selective law school, assuming its work remained constant throughout law school (and ignoring other graduation requirements that involve different skills). In an era where remote exam administration has become the norm, this could hypothetically result in a struggling law student using ChatGPT to earn a JD that does not reflect her abilities or readiness to practice law."