AI on deck: assessing impact of MLB’s new ball-strike system
ITHACA, N.Y. - For 150 years, Major League Baseball (MLB) players and fans have accepted that an umpire missing a few balls and strikes is just part of the game. But this spring, MLB is rolling out an artificial intelligence-augmented camera system that will provide a second opinion for players to tap if they think an umpire whiffed.
This historic change inspired a Cornell research team to study how MLB stakeholders are integrating the Automated Ball-Strike System (ABS) – which tracks pitches in real time – into baseball’s sacred gameplay.
“We hear so much about AI influencing political views and fueling polarization, and here’s a case of AI being used as a consensus-building platform rather than creating division,” said Waki Kamino, a doctoral student in the field of information science, who with fellow information science doctoral student Andrea Wen-Yi Wang and other colleagues has spent the last year attending spring training games and umpire trainings, and interviewing league executives, umpires and fans. “It’s such a cool thing to see.”
So far, Kamino, Wang and a team of human-robot interaction researchers from the Cornell Ann S. Bowers College of Computing and Information Science have published two papers, and submitted a third, exploring the tension that arises when technological precision is applied to the ambiguities of human decision-making.
Baseball’s ABS consists of 12 AI-powered, Hawk-Eye cameras installed in each stadium and all focused on the strike zone – the roughly 17-inch-wide space between the batter’s knees and chest. Trained and honed with umpires’ feedback, the ABS will get called up to the big leagues this year after seven seasons in the minor leagues, where it was used and refined in thousands of games.
In each game, teams can challenge an umpire’s ball or strike call, with only the pitcher, catcher or batter permitted to initiate those challenges. However, players must be judicious with challenges – if they lose two, the team is out of challenges for the rest of the game. Reviews will take about 15 seconds, and the Hawk-Eye pitch visualization will be shown on stadium video boards and to viewers at home.
Baseball’s strike zone offered a fascinating test case, Kamino said: How can technology determine balls and strikes when MLB’s very definition of the strike zone is about as clear as a Paul Skenes fastball is hittable?
“Ambiguity is core to MLB and to the idea of what makes a good game and a good experience,” said Kamino, who’s published research on social robots and the bonds robot owners form with them and each other. “The strike zone is ambiguous. It’s a social construct. How do you even automate that?”
That question struck Kamino two years ago after checking in on her favorite player, Shohei Ohtani, the otherworldly pitcher-slugger for the Los Angeles Dodgers. With the Dodgers game on, she heard the broadcasters discussing the impending rollout of “robot cameras.”
“We should study this,” Kamino texted Wang, whose research explores the use of AI systems to automate unclear definitions and concepts.
Then, Kamino took a big swing. She and Wang cold-emailed MLB Commissioner Rob Manfred ’80 and made a pitch to the Cornell ILR School graduate: Kamino and her co-researchers wanted to study the ABS to find out how a massive organization with so many stakeholders was implementing and making sense of an AI tool.
Manfred responded within 30 minutes and gave the green light. That eventually led to on-field research work last summer, when Kamino and Wang attended ABS-enabled spring training games in Arizona, umpire training camps and a baseball analytics conference.
“Introducing technology into baseball isn’t like bringing a robot into a manufacturing line,” said Malte Jung, associate professor of information science, the Nancy H. ’62 and Philip M. ’62 Young Sesquicentennial Faculty Fellow, and Kamino’s adviser and research collaborator. “You’re bringing technology into a game that has a culture and a history, with an audience in the millions. As researchers, what was compelling to us was the opportunity to study human-machine interaction in the wild at a scale we never considered.”
Implementing automated enforcement systems like the ABS into existing organizations requires complex consensus-building and sense-making among stakeholders, researchers found. After seven years of development and testing, the ABS appears ready for “The Show,” and that’s a credit to MLB’s careful rollout and willingness to adapt the system based on feedback, researchers added.
These findings were outlined in “Beyond Accuracy: Rethinking the Value of AI in Decision-Making Through Baseball’s Automated Ball-Strike (ABS) System,” which Kamino presented at the Hawaii International Conference on System Science (HICSS) on Jan. 7. The paper is authored by Kamino, Wang, Selma Sabanovic, professor in the Luddy School of Informatics, Computing and Engineering at Indiana University, and Jung.
In its newest ABS-related paper, “What is a Robot? Understanding Baseball’s ‘Robot Umpire’ Through the Lens of Fluid Technology,” Cornell researchers investigate why some fans and media call ABS a “robot umpire” in the first place. After all, it’s an inconspicuous camera system with no typical robotic features.
They argue that robots are better understood as “fluid, sociotechnical assemblages” rather than fixed hunks of hardware and software. Robots are adapted and shaped by the communities they serve, researchers said. The paper will be presented at the 21st annual Institute of Electrical and Electronics Engineers /Association for Computing Machinery’s International Conference on Human-Robot Interaction, held March 16-19 in Edinburgh, Scotland. The paper is authored by Kamino, Wang, Guy Hoffman, associate professor in the Sibley School of Mechanical and Aerospace Engineering, Sabanovic, and Jung.
Wang is the lead author of a third related paper, currently under peer review, showing that enforcement technologies like the ABS don’t simply enforce rules, but shape the rules. In the ABS’ case, it translates and thus redefines the strike zone from an imaginary 3D prism hovering above home plate into 2D, making it more readable for the ABS technology and viewer-friendly for visual presentation on broadcasts.
“Paradoxically, the ABS was brought in to enforce the rules, but it also changed the rules so that it would be accepted by multiple stakeholder groups,” Wang said. “This bidirectional relationship between enforcement technology and the rules is less talked about and reveals why we need to study technology from a system’s lens.”
Additionally, the use of ABS confirms what most baseball fans know but choose not to admit, researchers said.
“Umpires,” Kamino said, “are very good at their jobs.”
-30-
