A successful example of “adversarial collaboration.” When does this approach work and when does it not?
Stephen Ceci, Shulamit Kahn, and Wendy Williams write:
We synthesized the vast, contradictory scholarly literature on gender bias in academic science from 2000 to 2020. . . . Claims and counterclaims regarding the presence or absence of sexism span a range of evaluation contexts. Our approach relied on a combination of meta-analysis and analytic dissection. We evaluated the empirical evidence for gender bias in six key contexts in the tenure-track academy: (a) tenure-track hiring, (b) grant funding, (c) teaching ratings, (d) journal acceptances, (e) salaries, and (f) recommendation letters. We also explored the gender gap in a seventh area, journal productivity, because it can moderate bias in other contexts. . . . Contrary to the omnipresent claims of sexism in these domains appearing in top journals and the media, our findings show that tenure-track women are at parity with tenure-track men in three domains (grant funding, journal acceptances, and recommendation letters) and are advantaged over men in a fourth domain (hiring). For teaching ratings and salaries, we found evidence of bias against women; although gender gaps in salary were much smaller than often claimed, they were nevertheless concerning.
They continue:
Even in the four domains in which we failed to find evidence of sexism disadvantaging women, we nevertheless acknowledge that broad societal structural factors may still impede women’s advancement in academic science. . . . The key question today is, in which domains of academic life has explicit sexism been addressed? And in which domains is it important to acknowledge continuing bias that demands attention and rectification lest we maintain academic systems that deter the full participation of women? . . .
Our findings of some areas of gender neutrality or even a pro-female advantage are very much rooted in the most recent decades and in no way minimize or deny the existence of gender bias in the past. Throughout this article, we have noted pre-2000 analyses that suggested that bias either definitely or probably was present in some aspects of tenure-track academia before 2000. . . .
The authors characterize this project as an “adversarial collaboration”:
This article represents more than 4.5 years of effort by its three authors. By the time readers finish it, some may assume that the authors were in agreement about the nature and prevalence of gender bias from the start. However, this is definitely not the case. Rather, we are collegial adversaries who, during the 4.5 years that we worked on this article, continually challenged each other, modified or deleted text that we disagreed with, and often pushed the article in different directions. . . . Kahn has a long history of revealing gender inequities in her field of economics, and her work runs counter to Ceci and Williams’s claims of gender fairness. . . . In 2019, she co-organized a conference on women in economics, and her most recent analysis in 2021 found gender inequities persisting in tenure and promotion in economics. . . . Her findings diverge from Ceci and Williams’s, who have published a number of studies that have not found gender bias in the academy, such as their analyses of grants and tenure-track hiring . . .
Although our divergent views are real, they may not be evident to readers who see only what survived our disagreements and rewrites; the final product does not reveal the continual back and forth among the three of us. Fortunately, our viewpoint diversity did not prevent us from completing this project on amicable terms. Throughout the years spent working on it, we tempered each other’s statements and abandoned irreconcilable points, so that what survived is a consensus document that does not reveal the many instances in which one of us modified or cut text that another wrote because they felt it was inconsistent with the full corpus of empirical evidence. . . .
Editors and board members can promote science by encouraging, when possible, diverse viewpoints and by commissioning teams of adversarial coauthors (as this particular journal, Psychological Science in the Public Interest, was founded to do—to bring coauthors together in an attempt to resolve their historic differences). Knowing that one’s writing will be criticized by one’s divergently thinking coauthors can reduce ideologically driven criticisms that are offered in the guise of science. . . .
Interesting. In the past I’ve been suspicious of adversarial collaborations—whenever I’ve tried such a thing it hasn’t worked so well, and examples I’ve seen elsewhere have seemed to have more of the “adversarial” than the “collaboration.”
Here are two examples (here and here) where I tried to work with people who I disagreed with, but they didn’t want to work with me.
I get it: in both places I was pretty firm that they had been making strong claims that were not supported by their evidence, and there was no convenient halfway point where they could rest. Ideally they’d just have agreed with me, but it’s pretty rare that people will just give up something they’ve already staked a claim on.
I’m not saying these other researchers are bad people. In each case, there was a disagreement about the strength of evidence. My point is just that there was no clear way forward regarding an adversarial collaboration. So I just wrote my articles on my own; I consider each of these to be a form of “asynchronous collaboration.” Still better than nothing.
But this one by Ceci, Kahn, and Williams seems to have worked well. Perhaps it’s easier in psychology than in political science, for some reason?
That said, I can’t imagine a successful adversarial collaboration with the psychologists who published some of the horrible unreplicable stuff from the 2005-2020 era. They just seem too invested in their claims, also they achieved professional success with that work and have no particular motivation to lend their reputations to any work that might shoot it down. By their behavior, they treat their claims as fragile and would not want them to be put to the test. The Ceci, Kahn, Williams example is different, perhaps, because there are policy questions at stake, and all of them are motivated to persuade people in the middle of the debate. In contrast, the people pushing some of the more ridiculous results in embodied cognition and evolutionary psychology have no real motivation to persuade skeptics or even neutrals; they just need to keep their work from being seen as completely discredited.
This is related to my point about research being held to a higher standard when it faces active opposition.
No comments:
Post a Comment