More Thoughts on Reproducibility

I'm having some more thoughts about the series of

Anton Howes

posts I've read recently about the potential replication or reproducibility crisis in History and his suggestion that we adopt a policy of "Open History" where we publish digital collections of our sources. My first reaction to this suggestion is YES! That's a fabulous idea. There are growing numbers of online archives already, but I would particularly love it if there were digital collections of archival sources that particular historians had used. Maybe photos of manuscripts with text files of the transcriptions or translations that folks had done of them. These are often very time-consuming elements of research; but it's important to be able to see the originals to catch any errors in transcription. And then it's important to have the context of the full source, to help readers decide whether they agree with the way the historian interpreted it.

One of the things I wrote in my original response to the Howes post was that if an interpretation or a truth claim hangs on single piece of data, there may be something wrong. In science there's an idea that a single piece of disconfirming data is enough, while even a mountain of supporting data can't conclusively prove a proposition. I think it's a bit more complicated in history, since we're more often talking about interpretations than proofs; so a great piece of counter-evidence could cast doubt on an interpretation but it might take more information to support an alternative interpretation. On the other hand, I think most good historical theses do try to rely on mountains of data. I read thousands of handwritten nineteenth-century personal letters for Peppermint Kings and cited hundreds. My interpretations and arguments really came much more from the entire collection than from any particular letter. I can imagine a situation where a single piece of evidence might be very important. But even if you uncovered a letter from Allen Dulles to Lee Harvey Oswald telling him when to be at the Book Depository and that he'd have backup from the grassy knoll, you'd be super excited but you'd still try to confirm its reality. You'd want to make that a centerpiece but support it with more evidence.

It really is unfortunate that the past is so big that you can often get away with lying about it. There are plenty of examples of this, but I'm still not convinced this is history's biggest problem. It still strikes me that focusing on facts that people got wrong (accidentally or otherwise) suggests that history is mostly about getting the facts right. Reporting the past as Leopold von Ranke said, wie es eigentlich gewesen ist. I don't think that's fundamentally what we're doing, most of the time. I don't think that's the most interesting or exciting aspect of what we do.

That's why I'd be happy to create digital collections of my sources for people to peruse and why I'd love to see the collections other people have used to write their histories. I fully expect that I would see different things and maybe come to different interpretations and conclusions. We've discussed this in my book club recently, talking about Adler and Van Doren's How to Read a Book. Every reader's goals, interests, and backgrounds are different; their highlights, notes, and understanding of the book will reflect those differences. The idea that there is a single message in a book that it's our job to extract seems a bit magisterial and modern. Similarly, the idea that two researchers will find the same story or interpretation or "truth" in an archive seems absurd.

But yeah, I agree: we're not allowed to make shit up. And showing our work will help prevent that. It will also give readers a glimpse under the covers, which might help them appreciate the complexity and the work that's actually involved. That wouldn't be bad either.