Facial recognition AI helps save multibillion dollar grape crop

Grant and Award Announcement

CORNELL UNIVERSITY

ITHACA, N.Y. – New technology, using robotics and AI, is supercharging efforts to protect grape crops and will soon be available to researchers nationwide working on a wide array of plant and animal research.

Biologist Lance Cadle-Davidson, an adjunct professor in the School of Integrative Plant Science (SIPS) at Cornell University and a research plant pathologist with the U.S. Department of Agriculture’s Agricultural Research Service (USDA-ARS), is working to develop grape varieties that are more resistant to powdery mildew, which can show up in infrared before they are visible to the naked eye. But his lab’s research has been bottlenecked by the need to manually assess thousands of grape leaf samples for evidence of infection.

So Cadle-Davidson’s team developed prototypes of imaging robots that could scan grape leaf samples automatically – a process called high-throughput phenotyping – through the USDA-ARS funded VitisGen2 grape breeding project and in partnership with the Light and Health Research Center. This partnership led to the creation of a robotic camera they named “BlackBird.”

The BlackBird robot can gather information at a scale of 1.2 micrometers per pixel – equivalent to a regular optical microscope. For each 1-centimeter leaf sample being examined, the robot provides 8,000 by 5,000 pixels of information.

Extracting useful information from such a large, high-resolution image was the challenge for engineer and computer scientist Yu Jiang, an assistant research professor in SIPS’ Horticulture Section at Cornell AgriTech – and his team used AI to solve it.

Using breakthroughs in deep neural networks developed for computer vision tasks like face recognition, Jiang applied this knowledge to the analysis of microscopic images of grape leaves. In addition, Jiang and his team implemented the visualization of the network inferential processes, which help biologists better understand the analysis process and build confidence with AI models.

Working together, Cadle-Davidson’s team tests and validates what the robots see, enabling Jiang’s team to teach them how to identify biological traits more effectively. The results are astounding, Cadle-Davison said. Research experiments that used to take his entire lab team six months to complete now take the BlackBird robots just one day.

“It has revolutionized our science,” Cadle-Davidson said. “And we’re finding that Yu’s AI tools actually do a better job of explaining the genetics of these grapes than we can do sitting at a microscope for months at a time doing backbreaking work.”

In July the team was awarded a two-year, $150,000 grant from the Cornell Institute for Digital Agriculture Research Innovation Fund to begin upgrading the BlackBird robot to see beyond the red-green-blue color spectrum and into infrared.

If the researchers can develop tools to help farmers detect disease early, it would enable farmers to target fungicide sprays before infection spreads, meaning less fungicide and fewer lost crops. They’re also working to integrate AI more effectively with scientists in data analysis.

They were also awarded a $100,000 grant from the USDA-ARS to disseminate BlackBird to ARS field offices working on other crops that do the same kind of high-throughput phenotyping work.

“We hope to find collaborative labs who can join us in taking advantage of this tool,” Jiang said. “We see potential applications for this research in plant studies, animal fields or medical purposes.”

For additional information, see this Cornell Chronicle story.