Researchers unveil new cipher system that protects computers against spy programs

Meeting Announcement

TOHOKU UNIVERSITY

A group of international researchers has achieved a breakthrough in computer security with the development of a new and highly efficient cipher for cache randomization. The innovative cipher, designed by Assistant Professor Rei Ueno from the Research Institute of Electrical Communication at Tohoku University, addresses the threat of cache side-channel attacks, offering enhanced security and exceptional performance.

Cache side-channel attacks pose a significant threat to modern computer systems, as they can stealthily extract sensitive information, including secret keys and passwords, from unsuspecting victims. These attacks exploit vulnerabilities in the operating principles of contemporary computers, making their countermeasures extremely challenging. Cache randomization has emerged as a promising countermeasure; however, identifying a secure and effective mathematical function for this purpose has been a lingering challenge.

To overcome this, Ueno and his colleagues created SCARF. SCARF is based on a comprehensive mathematical formulation and modeling of cache side-channel attacks, offering robust security. Moreover, SCARF exhibits remarkable performance, completing the randomization process with only half the latency of existing cryptographic techniques. The cipher's practicality and performance were thoroughly validated through rigorous hardware evaluations and system-level simulations.

The team comprised members from Tohoku University, CASA at Ruhr University Bochum, and NTT Social Informatics Laboratories at Nippon Telegraph and Telephone Corporation. 

"We are thrilled to announce SCARF, a powerful tool in enhancing computer security," said Ueno. "Our innovative cipher is engineered to be compatible with various modern computer architectures, ensuring its widespread applicability and potential to bolster computer security significantly."

SCARF's potential impact extends beyond individual computers, as its implementation has the capacity to contribute to building a more secure information society. By mitigating cache side-channel attack vulnerabilities, SCARF takes a critical step towards safeguarding sensitive data and user privacy.

The paper detailing the development will be presented at the USENIX Security Symposium on August 9, 2023.

A schematic outlining how the new SCARF system operates.

CREDIT

Rei Ueno

Genome data rewrite the story of oat domestication in China

Genome analysis of 100 oat plants from around the world reveal that different oat varieties were developed in two different domestication events, challenging current plant research assumptions

Peer-Reviewed Publication

GIGASCIENCE

 

IMAGE: OAT, AVENA SATIVA, IS ONE OF THE TOP TEN CEREALS CROPS IN TERMS OF GLOBAL PRODUCTION view more 

CREDIT: PHOTO BY F. WELTER-SCHULTES

Oat is among the top ten cereal crop species in terms of global production. It can adapt to different climates, and farmers can grow it successfully even in harsh environments where other crops such as rice and corn fail. However, not all oat plants are the same. Based on their grains, two major oak varieties can easily be distinguished: hulled, grains that are covered in a non-edible husk, and naked, grains that have a soft outer casing that easily separates from the edible grain during threshing. To gain information on the origins of these different varieties, researchers in China have sequenced the genomes of over 100 oat plants from around the world. Their analyses indicate that, unlike what is the current belief — that the two varieties came from one domestication event, the hulled and naked oat were domesticated independently. The work is published in the Open Science journal GigaScience.

It is believed that the common oat (Avena sativa), which today is grown all over the world, was domesticated in Europe around 3,000 years ago. In contrast, the origins of naked oat, which today is grown mainly in China, remain unclear. Many researchers regard naked oat as a variant of hulled oat, speculating that a mutation occurred after hulled oat was introduced into China. However, new population genomic data generated and analyzed by the laboratory of Prof. Bing Han at Inner Mongolia Agricultural University (IMAU) tell a different story. 

Rather than being a variant of common oat that separated relatively recently, the authors estimate that hulled oat and naked oat diverged around 51,000 years ago. They therefore speculate that the two varieties were domesticated independently a long time ago, rather than one being a recent derivative of the other. The analyses in the study include a set of whole genome sequences, including 89 naked oat and 22 hulled oat plants, as well as four other closely related hexaploid species from around the world.

Additional findings in this study arising from a deeper analysis of this large data set support this view. For example, if naked oat split recently from hulled oat, geneticists expected to see traces of a population bottleneck in the naked oat, which would have reduced the genetic diversity in the naked oat population. However, the scientists found the opposite: in their data, the genetic diversity of naked oat is higher than that of hulled oat, not the other way around.

The overall picture emerging from the data still remains rather complex, Prof. Bing Han explains: “The breeding of naked oat in China has gone through phases, including the direct collection and utilization of landraces, cross-breeding between naked oat varieties, and cross-breeding of naked oat with hulled oat.” All of this can increase the intricacy of the findings, leaving a great deal more to discover about the genetic history of naked oat.

The findings in this work demonstrate the power of large-scale genome sequencing to better understand the domestication history of one of the major crop species that is feeding the world today.

Further Reading:

Nan J; Ling Y; An J; Wang T; Chai M; Fu J; Wang G; Yang C; Yang Y; Han: B (2023): "Genome resequencing reveals independent domestication and breeding improvement of naked oat" GigaScience;  https://doi.org/10.1093/gigascience/giad061

Article URL: https://academic.oup.com/gigascience/article-lookup/doi/10.1093/gigascience/giad061

Data Availability:

Nan J; Ling Y; An J; Wang T; Chai M; Fu J; Wang G; Yang C; Yang Y; Han: B (2023): Supporting data for "Genome resequencing reveals independent domestication and breeding improvement of naked oat" GigaScience Database. http://dx.doi.org/10.5524/102412

About GigaScience

GigaScience is co-published by GigaScience Press and Oxford University Press. Winner of the 2018 PROSE award for Innovation in Journal Publishing (Multidisciplinary), the journal covers research that uses or produces 'big data' from the full spectrum of the biological and biomedical sciences. It also serves as a forum for discussing the difficulties of and unique needs for handling large-scale data from all areas of the life and medical sciences. The journal has a completely novel publication format -- one that integrates manuscript publication with complete data hosting, and analyses tool incorporation. To encourage transparent reporting of scientific research as well as enable future access and analyses, it is a requirement of manuscript submission to GigaScience that all supporting data and source code be made available in the GigaScience database, GigaDB, as well as in publicly available repositories. GigaScience will provide users access to associated online tools and workflows, and has integrated a data analysis platform, maximizing the potential utility and re-use of data.

About GigaScience Press

GigaScience Press is BGI's Open Access Publishing division, which publishes scientific journals and data. Its publishing projects are carried out with international publishing partners and infrastructure providers, including Oxford University Press and River Valley Technologies. It currently publishes two award-winning data-centric journals: its premier journal GigaScience (launched in 2012), which won the 2018 American Publishers PROSE award for innovation in journal publishing, and its new journal GigaByte (launched 2020), which won the 2022 ALPSP Award for Innovation in Publishing. The press also publishes data, software, and other research objects via its GigaDB.org database. To encourage transparent reporting of scientific research and to enable future access and analyses, it is a requirement of manuscript submission to all GigaScience Press journals that all supporting data and source code be made openly available in GigaDB or in a community approved, publicly available repository.

---

JOURNAL

GigaScience

DOI

10.1093/gigascience/giad061 

METHOD OF RESEARCH

Experimental study

SUBJECT OF RESEARCH

Not applicable

ARTICLE TITLE

Genome resequencing reveals independent domestication and breeding improvement of naked oat

ARTICLE PUBLICATION DATE

1-Aug-2023

 

Nuisance vegetation removal in Senegalese waterways reduces the overall prevalence of parasitic infections and increases local food production

Peer-Reviewed Publication

UNIVERSITY OF CALIFORNIA - SANTA BARBARA

 

IMAGE: UCSB GEOGRAPHER DAVID LÓPEZ-CARR, CENTER LEFT, AND STANFORD UNIVERSITY HEALTH AND ENVIRONMENTAL SCIENTIST ANDREA LUND, CENTER RIGHT, WORKING WITH SENEGALESE PARTNERS view more 

CREDIT: COURTESY CREDIT

It’s an elegant solution: Remove the habitat of a parasite-carrying aquatic snail and reduce the level of infection in the local community; all while generating more feed and compost for local farmers.

A collaboration of scientists from the United States and Senegal focused on doing just that by removing overgrown aquatic vegetation from areas upstream of the Diama Dam in northeastern Senegal. In doing so, they generated positive impacts to the local communities’ health and economies.

 “It is rare and gratifying when we can find a potential win-win solution to both human health and livelihoods,” said UC Santa Barbara geography professor David López-Carr, a co-author of a paper that appears in the journal Nature. In it, the researchers provide proof for a hypothesis that agricultural activities, including the use of fertilizers, contribute to parasitic infections by fueling the growth of aquatic vegetation. “The results suggest a simple solution to positively impact society at the intersections of health, society and economy of northern Senegal, with implications for the over 700 million people globally in schistosomiasis endemic areas.”

Since the construction of the Diama Dam in 1986, local farmers have had better access to fresh water to irrigate their fields. However, the presence of the new infrastructure also has increased the prevalence of the schistosoma parasite, a tiny freshwater flatworm commonly found in Africa, South America and Southeast Asia. Nearly 250 million people around the world are estimated to be infected with this parasite.

 As far as tropical diseases go, schistosomiasis (also known as bilharzia or snail fever) isn’t immediately fatal or even transmissible between people. But in the long term, the condition is debilitating.

“The disease is most prevalent in poor communities lacking potable water and adequate sanitation,” said López-Carr, an anthropogeographer who specializes in human-environment dynamics in the developing world. Adult worms take up residence in blood vessels and lay eggs in tissue, causing reactions and generally wreaking havoc on organs. Long-term effects include increased risk for cancer and infertility, and those infected are less able to work and go to school, keeping them in the cycle of poverty. “Poor farmers can lose up to half of their yields due to infection,” he said.

Health agencies and organizations have been fighting these infections with drugs that work well, however, the medicine does not prevent reinfection, which can happen as soon as the individual encounters contaminated water. Previous research has also focused on using the snails’ natural predators — prawns — which were cut off by the dam.

In their effort to get ahead of the disease, the collaboration took a close look at the habitat that supports the worms’ intermediate host, a small snail that lives in the Senegal River and its tributaries. They found that a common aquatic plant called Ceratophyllum demersum — also known as hornwort — can hold up to 99% of these snails, with which they have a mutualistic relationship.

Exacerbated by fertilizer runoff from agricultural operations farther upstream, c. demersum and other aquatic plants tend to proliferate in local waterways, which impedes access for daily activities such as cooking, irrigation and washing clothes.

For their experiment, the researchers conducted a three-year randomized control trial in 16 communities, to see if and how much nuisance vegetation removal in about half of the communities would affect the presence of the snails. They measured baseline infection rates, administered antiparasitic drugs, removed the vegetation and then measured reinfection rates in more than 1,400 schoolchildren. In total, the research teams took out an estimated 430 metric tons (wet) of aquatic vegetation from water access points.

“In our randomized controlled trial, control sites — places where we didn’t remove submerged vegetation from water access points — had 124% higher intestinal schistosoma reinfection rates,” López-Carr said. In addition to lowered infection rates where they removed the vegetation, the researchers found that the removed material could be used to feed livestock, or turned into compost for growing crops, lowering costs dramatically and increasing yields for local farmers. In this way, according to López-Carr “the approach yielded an economic incentive to remove nuisance vegetation from waterways and return nutrients from aquatic plants back to the soil and for livestock feed with the promise of severing poverty-disease traps while lowering infectious burden at the same time.”

“A broader benefit is the hope that this example can set for enhancing win-win planetary health research and solutions that improve livelihoods while also reducing infectious morbidity and mortality,” he added.

 Having conducted these trials, the researchers hope that this study is implemented elsewhere in other similar regions to replicate the same kind of health and economic outcomes.

 And, it might not be just a solution for developing countries. “Perhaps vegetation growth resulting from excess nutrients could also be used as livestock feed in more developed countries as well,” López-Carr said.

---

JOURNAL

Nature

ARTICLE TITLE

A planetary health innovation for disease, food and water challenges in Africa

When electrons slowly vanish during cooling

Researchers observe an effect in the quantum world that does not exist in the macrocosm

Peer-Reviewed Publication

UNIVERSITY OF BONN

 

IMAGE: COMPOSED OF LOCALIZED AND MOBILE ELECTRONS, HERE BROKEN UP BY AN ULTRASHORT LIGHT PULSE. view more 

CREDIT: IMAGE: UNIVERSITY OF BONN

Many substances change their properties when they are cooled below a certain critical temperature. Such a phase transition occurs, for example, when water freezes. However, in certain metals there are phase transitions that do not exist in the macrocosm. They arise because of the special laws of quantum mechanics that apply in the realm of nature’s smallest building blocks. It is thought that the concept of electrons as carriers of quantized electric charge no longer applies near these exotic phase transitions. Researchers at the University of Bonn and ETH Zurich have now found a way to prove this directly. Their findings allow new insights into the exotic world of quantum physics. The publication has now been released in the journal Nature Physics.

If you cool water below zero degrees Celsius, it solidifies into ice. In the process, it abruptly changes its properties. As ice, for example, it has a much lower density than in a liquid state - which is why icebergs float. In physics, this is referred to as a phase transition.

But there are also phase transitions in which characteristic features of a substance change gradually. If, for example, an iron magnet is heated up to 760 degrees Celsius, it loses its attraction to other pieces of metal - it is then no longer ferromagnetic, but paramagnetic. However, this does not happen abruptly, but continuously: The iron atoms behave like tiny magnets. At low temperatures, they are oriented parallel to each other. When heated, they fluctuate more and more around this rest position until they are completely randomly aligned, and the material loses its magnetism completely. So while the metal is being heated, it can be both somewhat ferromagnetic and somewhat paramagnetic.

Matter particles cannot be destroyed

The phase transition thus takes place gradually, so to speak, until finally all the iron is paramagnetic. Along the way, the transition slows down more and more. This behavior is characteristic of all continuous phase transitions. “We call it ‘critical slowing down,’“ explains Prof. Dr. Hans Kroha of the Bethe Center for Theoretical Physics at the University of Bonn. “The reason is that with continuous transitions, the two phases get energetically closer and closer together.” It is similar to placing a ball on a ramp: It then rolls downhill, but the smaller the difference in altitude, the more slowly it rolls. When iron is heated, the energy difference between the phases decreases more and more, in part because the magnetization disappears progressively during the transition.

Such a “slowing down” is typical for phase transitions based on the excitation of bosons. Bosons are particles that “generate” interactions (on which, for example, magnetism is based). Matter, on the other hand, is not made up of bosons but of fermions. Electrons, for example, belong to the fermions.

Phase transitions are based on the fact that particles (or also the phenomena triggered by them) disappear. This means that the magnetism in iron becomes smaller and smaller as fewer atoms are aligned in parallel. “Fermions, however, cannot be destroyed due to fundamental laws of nature and therefore cannot disappear,” Kroha explains. “That’s why normally they are never involved in phase transitions.”

Electrons turn into quasi-particles

Electrons can be bound in atoms; they then have a fixed place which they cannot leave. Some electrons in metals, on the other hand, are freely mobile - which is why these metals can also conduct electricity. In certain exotic quantum materials, both varieties of electrons can form a superposition state. This produces what are known as quasiparticles. They are, in a sense, immobile and mobile at the same timetime – a feature that is only possible in the quantum world. These quasiparticles - unlike “normal” electrons - can be destroyed during a phase transition. This means that the properties of a continuous phase transition can also be observed there, in particular, critical slowing down.

So far, this effect could be observed only indirectly in experiments. Researchers led by theoretical physicist Hans Kroha and Manfred Fiebig’s experimental group at ETH Zurich have now developed a new method, which allows direct identification of the collapse of quasiparticles at a phase transition, in particular the associated critical slowing down.

“This has enabled us to show for the first time directly that such a slowdown can also occur in fermions,” says Kroha, who is also a member of the Transdisciplinary Research Area “Matter” at the University of Bonn and the Cluster of Excellence “Matter and Light for Quantum Computing” of the German Research Foundation. The result contributes to a better understanding of phase transitions in the quantum world. On the long term, the findings might also be useful for applications in quantum information technology.

Participating institutions and funding:
The study was carried out in collaboration of ETH Zurich and the University of Bonn. The work was funded by the Swiss National Science Foundation (SNF) and the German Research Foundation (DFG).

Publication: Chia-Jung Yang, Kristin Kliemt, Cornelius Krellner, Johann Kroha, Manfred Fiebig and Shovon Pal: Critical slowing down near a magnetic quantum phase transition with fermionic breakdown. Nature Physics, DOI: 10.1038/s41567-023-02156-7; Internet: https://www.nature.com/articles/s41567-023-02156-7

---

JOURNAL

Nature Physics

DOI

10.1038/s41567-023-02156-7 

METHOD OF RESEARCH

Experimental study

SUBJECT OF RESEARCH

Not applicable

ARTICLE TITLE

Critical slowing down near a magnetic quantum phase transition with fermionic breakdown

ARTICLE PUBLICATION DATE

31-Jul-2023

Why you shouldn’t declaw tigers or other big cats

Peer-Reviewed Publication

NORTH CAROLINA STATE UNIVERSITY

Declawing house cats to keep them from scratching people and furniture is controversial – and even banned in some countries and areas in the U.S. – but the practice is not limited to house cats. In a new study, researchers looked at the effects of declawing on larger cat species and found that declawing disproportionately impacts their muscular capabilities as compared to their smaller brethren.

While it is illegal in the U.S. to surgically modify an exotic animal, declawing is still done on large cats like lions and tigers, often in an effort to allow cubs to more safely be handled in photo opportunities or for entertainment purposes.

“What people might not realize is that declawing a cat is not like trimming our fingernails; rather, it is removing part or all of the last bone of each digit,” says Adam Hartstone-Rose, professor of biological sciences at North Carolina State University and corresponding author of the research. “Like us, each cat finger has three bones, and declawing is literally cutting that third bone off at the joint.”

The researchers looked at the muscular anatomy of over a dozen exotic cats – from smaller species including bobcats, servals and ocelots, to lions and tigers – to determine the effect of declawing on their forelimb musculature.

They measured muscle density and mass, and also examined muscle fibers from both clawed and declawed exotic cats. They found that for the larger species declawing resulted in 73% lighter musculature in the forearm’s digital flexors. These muscles are involved in unsheathing the claws. They also found that overall, forelimb strength decreased by 46% to 66%, depending on the size of the animal, and that other muscles in the forelimb did not compensate for these reductions.

“When you think about what declawing does functionally to a housecat, you hear about changes in scratching, walking or using the litter box,” says Lara Martens, NC State undergraduate student and lead author of the research. “But with big cats, there’s more force being put through the paws. So if you alter them, it is likely that the effects will be more extreme.”

This is because paw size and body mass don’t scale up at a 1:1 ratio. Paw area increases at a slower rate than does body mass (which is proportional to volume), so larger cats have smaller feet relative to their body size, and their paws must withstand more pressure.

“Additionally, big cats are more reliant on their forelimbs – they bear most of the weight, and these bigger cats use their forelimbs to grapple because they hunt much larger prey,” Martens says. “So biomechanically speaking, declawing has a more anatomically devastating effect in larger species.”

“As scientists, it is our job to objectively document the effects of this surgery on the animals,” Hartstone-Rose says, “but it is hard to ignore the cruelty of this practice. These are amazing animals, and we should not be allowed to cripple them, or any animals, in this way.”

The work appears in Animals. NC State undergraduates Sarah Piersanti, Arin Berger, and Nicole Kida, and Ph.D. student Ashley Deutsch, also contributed to the research. The work was done in partnership with colleagues from Carolina Tiger Rescue, a sanctuary that rescues exotic carnivores, especially big cats, who have often been neglected or mistreated.

-peake-

Note to editors: An abstract follows.

“The effects of onychectomy (declawing) on antebrachial myology across the full body size range of exotic species of Felidae”

DOI: 10.3390/ani13152462

Authors: Lara L. Martens, Sarah Jessica Piersanti, Arin Berger, Nicole A. Kida, Ashley R. Deutsch, Adam Harstone-Rose, North Carolina State University; Kathryn Bertok, Lauren Humphries, Angela Lassiter, Carolina Tiger Rescue, Pittsboro, North Carolina
Published: July 30, 2023 in Animals

Abstract:
While people are familiar with the practice of declawing domestic cats, “onychectomy” is also performed on non-domesticated species, including pantherines to prolong their use in entertainment opportunities. Although the surgery (the partial or complete removal of the distal phalanx) has clear osteological implications, its myological effects have never been studied. Because the mass of an animal increases cubically as a product of its volume and, the areas of its paws only increase as a square, larger felids have higher foot pressures and therefore the surgery may have particularly substantial functional effects for larger cats. In this study, we evaluate the forearms of clawed and declawed non-domestic felid specimens spanning the body size range of the whole family to evaluate the effects of onychectomy on muscle fiber architecture. We found that the deep digital flexors (the muscles most directly affected by onychectomy) of declawed felids are significantly lighter (~73%) and less powerful (46-66%), and that other muscles do not make up for these reductions. Thus, onychectomy has a substantial effect on the myological capabilities of cats and because these deficiencies are not compensated for in biomechanical disadvantaged larger felids, it is probably functionally even more devastating for these species.

---

JOURNAL

Animals

DOI

10.3390/ani13152462 

METHOD OF RESEARCH

Observational study

SUBJECT OF RESEARCH

Animals

ARTICLE TITLE

The Effects of Onychectomy (Declawing) on Antebrachial Myology across the Full Body Size Range of Exotic Species of Felidae

ARTICLE PUBLICATION DATE

30-Jul-2023

COI STATEMENT

The authors declare no conflict of interest.

 

Researchers discover method to overcome antimicrobial resistance

Peer-Reviewed Publication

UNIVERSITY OF OKLAHOMA

The World Health Organization has labeled antimicrobial resistance a global threat because most clinical antibiotics are no longer effective against certain pathogenic bacteria. The Center for Antibiotic Discovery and Resistance at the University of Oklahoma, led by Helen Zgurskaya, Ph.D., and Valentin Rybenkov, Ph.D., is working on finding alternative therapeutic solutions.

Antibiotics work by targeting specific parts of a bacteria cell, such as the cell wall or its DNA. Bacteria can become resistant to antibiotics in a number of ways, including by developing efflux pumps – proteins that are located on the surface of the bacteria cell. When an antibiotic enters the cell, the efflux pump pumps it out of the cell before it can reach its target so that the antibiotic is never able to kill the bacteria.

However, OU researchers have contributed to a recent discovery published in the journal Nature Communications. The scientists found a new class of molecules that inhibit the efflux pump and make the antibiotic effective again.

The inhibitors have a novel mechanism of action, which until recently remained unclear. Zgurskaya’s team, in collaboration with teams at the Georgia Institute of Technology and King’s College London in the United Kingdom, have uncovered that these inhibitors work as a “molecular wedge” that targets the area between the inner and outer cell membranes and increases antibacterial activities of antibiotics. Understanding this mechanism can facilitate the discovery of new therapeutics for clinical applications.

“We already live in a post-antibiotic era, and things will get much worse unless new solutions are found for antibiotic resistance in clinics. The discoveries we’ve made will facilitate the development of new treatments to help mitigate an impending crisis,” Zgurskaya said.

Helen Zgurskaya is a George Lynn Cross Research Professor and Valentin Rybenkov is a professor of biochemistry, both in the Department of Chemistry and Biochemistry, Dodge Family College of Arts and Sciences at the University of Oklahoma. Learn more about their research at the Center for Antibiotic Discovery and Resistance.

---

JOURNAL

Nature Communications

DOI

10.1038/s41467-023-39615-x 

ARTICLE TITLE

Conformational restriction shapes the inhibition of a multidrug efflux adaptor protein

 

Researchers find little evidence of cheating with online, unsupervised exams

Peer-Reviewed Publication

IOWA STATE UNIVERSITY

 

IMAGE: STUDENTS WORK ON LAPTOPS ABOVE “GENE POOL,” A TILE MOSAIC BY ANDREW LEICESTER INSIDE THE MOLECULAR BIOLOGY BUILDING AT IOWA STATE UNIVERSITY. view more 

CREDIT: PHOTO BY CHRISTOPHER GANNON/IOWA STATE UNIVERSITY.

AMES, IA — When Iowa State University switched from in-person to remote learning halfway through the spring semester of 2020, psychology professor Jason Chan was worried. Would unsupervised, online exams unleash rampant cheating?

His initial reaction flipped to surprise as test results rolled in. Individual student scores were slightly higher but consistent with their results from in-person, proctored exams. Those receiving B’s before the COVID-19 lockdown were still pulling in B’s when the tests were online and unsupervised. This pattern held true for students up and down the grading scale.

“The fact that the student rankings stayed mostly the same regardless of whether they were taking in-person or online exams indicated that cheating was either not prevalent or that it was ineffective at significantly boosting scores,” says Chan.

To know if this was happening at a broader level, Chan and Dahwi Ahn, a Ph.D. candidate in psychology, analyzed test score data from nearly 2,000 students across 18 classes during the spring 2020 semester. Their sample ranged from large, lecture-style courses with high enrollment, like introduction to statistics, to advanced courses in engineering and veterinary medicine.

Across different academic disciplines, class sizes, course levels and test styles (i.e., predominantly multiple choice or short answer), the researchers found the same results. Unsupervised, online exams produced scores very similar to in-person, proctored exams, indicating they can provide a valid and reliable assessment of student learning.

The research findings were recently published in Proceedings of the National Academy of Sciences.

“Before conducting this research, I had doubts about online and unproctored exams, and I was quite hesitant to use them if there was an option to have them in-person. But after seeing the data, I feel more confident and hope other instructors will, as well,” says Ahn.

Both researchers say they’ve continued to give exams online, even for in-person classes. Chan says this format provides more flexibility for students who have part-time jobs or travel for sports and extra-curriculars. It also expands options for teaching remote classes. Ahn led her first  online course over the summer.

Why might cheating have had a minimal effect on test scores?

The researchers say students more likely to cheat might be underperforming in the class and anxious about failing. Perhaps they’ve skipped lectures, fallen behind with studying or feel uncomfortable asking for help. Even with the option of searching Google during an unmonitored exam, students may struggle to find the correct answer if they don’t understand the content. In their paper, the researchers point to evidence from previous studies comparing test scores from open-book and close-book exams.

Another factor that may deter cheating is academic integrity or a sense of fairness, something many students value, says Chan. Those who have studied hard and take pride in their grades may be more inclined to protect their exam answers from students they view as freeloaders.

Still, the researchers say instructors should be aware of potential weak spots with unsupervised, online exams. For example, some platforms have the option of showing students the correct answer immediately after they select a multiple-choice option. This makes it much easier for students to share answers in a group text.

To counter this and other forms of cheating, instructors can:

• Wait to release exam answers until the test window closes.
• Use larger, randomized question banks.
• Add more options in multiple-choice questions and making the right choice less obvious.
• Adjust grade cutoffs.

COVID-19 and ChatGPT

Chan and Ahn say the spring 2020 semester provided a unique opportunity to research the validity of online exams for student evaluations. However, there were some limitations. For example, it wasn’t clear what role stress and other COVID-19-related impacts may have played on students, faculty and teaching assistants. Perhaps instructors were more lenient with grading or gave longer windows of time to complete exams.

The researchers said another limitation was not knowing if the 18 classes in the sample normally get easier or harder as the semester progresses. In an ideal experiment, half of the students would have taken online exams for the first half of the semester and in-person exams for the second half.

They attempted to account for these two concerns by looking at older test score data from a subset of the 18 classes during semesters when they were fully in-person. The researchers found the distribution of grades in each class was consistent with the spring 2020 semester and concluded that the materials covered in the first and second halves of the semester did not differ in their difficulty.

At the time of data collection for this study, ChatGPT wasn’t available to students. But the researchers acknowledge AI writing tools are a gamechanger in education and could make it much harder for instructors to evaluate their students. Understanding how instructors should approach online exams with the advent of ChatGPT is something Ahn intends to research.

The study was supported by a National Science Foundation Science of Learning and Augmented Intelligence Grant.

---

JOURNAL

Proceedings of the National Academy of Sciences

DOI

10.1073/pnas.2302020120 

METHOD OF RESEARCH

Data/statistical analysis

SUBJECT OF RESEARCH

People

ARTICLE TITLE

Unproctored online exams provide meaningful assessment of student learning

That’s funny – but AI models don’t get the joke

Reports and Proceedings

CORNELL UNIVERSITY

ITHACA, N.Y. -- Large neural networks, a form of artificial intelligence, can generate thousands of jokes along the lines of “Why did the chicken cross the road?” But do they understand why they’re funny?

Using hundreds of entries from the New Yorker magazine’s Cartoon Caption Contest as a testbed, researchers challenged AI models and humans with three tasks: matching a joke to a cartoon; identifying a winning caption; and explaining why a winning caption is funny. 

In all tasks, humans performed demonstrably better than machines, even as AI advances such as ChatGPT have closed the performance gap. So are machines beginning to “understand” humor? In short, they’re making some progress, but aren’t quite there yet.

“The way people challenge AI models for understanding is to build tests for them – multiple choice tests or other evaluations with an accuracy score,” said Jack Hessel, Ph.D. ’20, research scientist at the Allen Institute for AI (AI2). “And if a model eventually surpasses whatever humans get at this test, you think, ‘OK, does this mean it truly understands?’ It’s a defensible position to say that no machine can truly `understand’ because understanding is a human thing. But, whether the machine understands or not, it’s still impressive how well they do on these tasks.”

Hessel is lead author of “Do Androids Laugh at Electric Sheep? Humor ‘Understanding’ Benchmarks from The New Yorker Caption Contest,” which won a best-paper award at the 61st annual meeting of the Association for Computational Linguistics, held July 9-14 in Toronto.

Lillian Lee ’93, the Charles Roy Davis Professor in the Cornell Ann S. Bowers College of Computing and Information Science, and Yejin Choi, Ph.D. ’10, professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, and the senior director of common-sense intelligence research at AI2, are also co-authors on the paper.

For their study, the researchers compiled 14 years’ worth of New Yorker caption contests – more than 700 in all. Each contest included: a captionless cartoon; that week’s entries; the three finalists selected by New Yorker editors; and, for some contests, crowd quality estimates for each submission.  

For each contest, the researchers tested two kinds of AI – “from pixels” (computer vision) and “from description” (analysis of human summaries of cartoons) – for the three tasks.

“There are datasets of photos from Flickr with captions like, ‘This is my dog,’” Hessel said. “The interesting thing about the New Yorker case is that the relationships between the images and the captions are indirect, playful, and reference lots of real-world entities and norms. And so the task of ‘understanding’ the relationship between these things requires a bit more sophistication.”

In the experiment, matching required AI models to select the finalist caption for the given cartoon from among “distractors” that were finalists but for other contests; quality ranking required models to differentiate a finalist caption from a nonfinalist; and explanation required models to generate free text saying how a high-quality caption relates to the cartoon.

Hessel penned the majority of human-generated explanations himself, after crowdsourcing the task proved unsatisfactory. He generated 60-word explanations for more than 650 cartoons.

“A number like 650 doesn’t seem very big in a machine-learning context, where you often have thousands or millions of data points,” Hessel said, “until you start writing them out.”

This study revealed a significant gap between AI- and human-level “understanding” of why a cartoon is funny. The best AI performance in a multiple choice test of matching cartoon to caption was only 62% accuracy, far behind humans’ 94% in the same setting. And when it came to comparing human- vs. AI-generated explanations, humans’ were preferred roughly 2-to-1.

While AI might not be able to “understand” humor yet, the authors wrote, it could be a collaborative tool humorists could use to brainstorm ideas.

Other contributors include Ana Marasovic, assistant professor at the University of Utah School of Computing; Jena D. Hwang, research scientist at AI2; Jeff Da, research assistant at the University of Washington Rowan Zellers, researcher at OpenAI; and humorist Robert Mankoff, president of Cartoon Collections and long-time cartoon editor at the New Yorker.

The authors wrote this paper in the spirit of the subject matter, with playful comments and footnotes throughout.

“This three or four years of research wasn’t always super fun,” Lee said, “but something we try to do in our work, or at least in our writing, is to encourage more of a spirit of fun.”

This work was funded in part by the Defense Advanced Research Projects Agency; AI2; and a Google Focused Research Award.

-30-