It’s law review season, so I’ve been having a lot of conversations both on and off social media lately about the student-run law review system. Interesting, for example, is this recent survey by Professor Derek Muller.
The quality of law review articles in major law journals is ___ compared to ten years ago.
— Derek T.Muller (@derektmuller) February 22, 2024
I think it’s a fact that everyone thinks academic journals should publish better scholarship and not publish worse scholarship. But reflecting on Professor Muller’s question, it seems important to me to distinguish between the two different types of errors that a journal can commit: false positives (i.e. publishing something bad) and false negatives (i.e. refusing to publish something good).
Since no system is perfect, there is some tension between these two goals. If you want to emphasize the need to avoid false positives, then you probably want a field in which there are a small number of highly regarded journals with rigorous peer review based on rigorous methods. Many potentially “good” articles might be excluded from this system, but if something gets published in the industry’s top journal, you can pretty much take it to the bank.
On the other hand, if you want to avoid false negatives, then you would probably want a system closer to the current law review system, with many journals applying a much more pluralistic conception of merit, quickly chasing pieces through submission simultaneous. Most meaningful and relevant pieces can find a decent home.
Now neither system is perfect even at achieving the goal it is trying to maximize: there are rumors of corruption even in the most rigorously peer-reviewed fields, and there are still excellent law firms that somehow don’t fit the trend of the student editors. . (And for the latter case may I again recommend the Journal of Legal Analysis, a peer-reviewed law journal at Harvard Law School, where I serve as a co-editor, particularly for public law articles?).
But I think it’s helpful to articulate this distinction and these tradeoffs, and to remember that some changes that would solve one of these problems would make the other much worse. (For example, returning to Muller’s poll question, my guess is that the law review system, taken as a whole, has perhaps gotten slightly better at avoiding false negatives (i.e., finding homes for the good bits.) although perhaps it’s gotten slightly worse at avoiding false negatives (i.e. letting bad bits into good forums).
[For previous posts on law reviews, see here and here.]