Machine learning has a flaw; it's gullible
Research shows how humans can shield machine learning from manipulation
Machine learning (ML) - technology in which algorithms "learn" from existing patterns in data to conduct statistically driven predictions and facilitate decisions - has been found in multiple contexts to reveal bias. An example is Amazon.com coming under fire for a hiring algorithm that revealed gender and racial bias. Such biases often result from slanted training data or skewed algorithms.
And in other business contexts, there's another potential source of bias. It comes when outside individuals stand to benefit from bias predictions, and work to strategically alter the inputs. In other words, they're gaming the ML systems. A couple of the most common contexts are perhaps job applicants and people making a claim against their insurance. ML algorithms are built for these contexts. They can review resumes faster than any recruiter and comb through insurance claims faster than any human processor. But people who submit resumes and insurance claims have a strategic interest in getting positive outcomes - and some of them know how to outthink the algorithm.
Rajshree Agarwal and Evan Starr, researchers at the University of Maryland's Robert H. Smith School of Business worked with Prithwiraj Choudhury at Harvard Business School to set out to answer "Can ML correct for such strategic behavior?"
They found that two critical attributes of humans serve as important complements to machine learning in correcting for these biases. The more obvious one is vintage-specific skills, this ensures that humans have the ability to properly interface with machine learning so they can guide it appropriately. The other important attribute is domain-specific expertise, the knowledge that humans can provide machines on how to correct for the incompleteness of inputs.
Their work is forthcoming in Strategic Management Journal as "Machine Learning and Human Capital Complementarities: Experimental Evidence on Bias Mitigation."
Prior research in so-called "adversarial" ML looked closely at attempts to "trick" ML technologies, and generally concluded that it's extremely challenging to prepare the ML technology to account for every possible input and manipulation. In other words, ML is trickable.
What should firms do about it? Can they limit ML prediction bias? And, is there a role for humans to work with ML to do so? Agarwal, Choudhury and Starr honed their focus on patent examination, a context rife with potential trickery. "Patent examiners face a time-consuming challenge of accurately determining the novelty and nonobviousness of a patent application by sifting through ever-expanding amounts of 'prior art,'" or inventions that have come before, the researchers explain. It's challenging work.
Compounding the challenge: patent applicants are permitted by law to create hyphenated words and assign new meaning to existing words to describe their inventions. It's an opportunity, the researchers explain, for applicants to strategically write their applications in a strategic, ML-targeting way. The U.S. Patent and Trademark Office is generally wise to this. It has invited in ML technology that "reads" the text of applications, with the goal of spotting the most relevant prior art quicker and leading to more accurate decisions. "Although it is theoretically feasible for ML algorithms to continually learn and correct for ways that patent applicants attempt to manipulate the algorithm, the potential for patent applicants to dynamically update their writing strategies makes it practically impossible to train an ML algorithm to correct for this behavior," the researchers write.
In its study, the team conducted observational and experimental research. They found that patent language changes over time, making it highly challenging for any ML tool to operate perfectly on its own. The ML benefited strongly, they found, from human collaboration.
People with skills and knowledge accumulated through prior learning within a domain complement ML in mitigating bias stemming from applicant manipulation, the researchers found, because domain experts bring relevant outside information to correct for strategically altered inputs. And individuals with vintage-specific skills - skills and knowledge accumulated through prior familiarity of tasks with the technology - are better able to handle the complexities in ML technology interfaces.
They caution that although the provision of expert advice and vintage-specific human capital increases initial productivity, it remains unclear whether constant exposure and learning-by-doing by workers would cause the relative differences between the groups to grow or shrink over time. They encourage further research into the evolution in the productivity of all ML technologies, and their contingencies.
###
The research drew the 2019 Best Conference Paper Award from the Strategic Management Society and won the Best Interdisciplinary Paper Award from the Strategic Human Capital Interest Group at Strategic Management Society 2019.
No comments:
Post a Comment