Apr 2, 2013

The XL-Files: If You Think Statistics is Hard, Try History…

Contributing Editor Xiao-Li Meng writes:

“Upstairs on the Square” is reputed in some circles to be the restaurant for dining and wining around Harvard Square. The probability therefore was not surprisingly small to find Ben Affleck and two statisticians there sitting only a few tables apart. It was September 15, 2009, and The Town was in town, or rather in the Square. The director and actor was busy conducting a working lunch, talking attentively to (or directing?) his leading lady.

Alan Agresti was in town as well, as a visiting professor. It was the first opportunity for me to raise a glass to Alan to thank him for teaching for us despite being on permanent sabbatical (Alan’s phrase for retirement). Neither of us, however, anticipated at that moment that, just as Ben Affleck’s lunch was to make a movie, ours was about to make history.

At some point Alan mentioned the upcoming fiftieth anniversary for Wisconsin Statistics in 2010, and I reflected upon the fiftieth anniversary celebration of my department during 2007. As we lamented the loss of some founding members and their memories, it dawned on us that we ought to do something to reduce the losses. After all, history is data, and data are snapshots of history.

However, although both Alan and I are seasoned statisticians, when it came to conducting a history project, we inevitably fell into the how-hard-can-it-be novice trap. We did discuss how time consuming such a project could be, but I somehow convinced myself—and apparently Alan—that if we asked each department to contribute a chapter, our job would simply be to put them together. How hard could that be?

Any reputable historian would be laughing aloud at such a naive attitude towards documenting history, just as we statisticians would laugh at a historian for thinking that analyzing a historical data set simply means running it through statistical software. But as a Chinese proverb has it, “newborn calves are unafraid of tigers,” we pressed on. Or I should say Alan pressed on, for my overestimation of my ability became apparent very soon. With Alan’s pursuit and persuasion, all 39 statistics departments from US universities that have (or had) a major history rooted more than 45 years ago agreed to participate: a 100% response rate!

The real success came only after intense work by all 39 sets of authors. Just as collecting data scientifically is never easy, summarizing fairly more than 45 years of a departmental history in a chapter of 5000–8000 words is a daunting, if not impossible, task. I learned this the hard way when I worked on my chapter. Even just restricted to the six volumes of documents meticulously kept by our founding father, Fred Mosteller, what would be my guiding principles to select the materials that would represent the departmental history in a fair way? Indeed, what kind of fairness should I aim to achieve?

Thinking statistically, the notion of representativeness arises naturally. But what does this mean in the context of selecting historical events? Clearly there is no meaningful “frame” of historical events to sample from. And which one is more fair? Give every former or current member an equal amount of space, or assign space in proportion to their contributions? And what contributions should be considered, and how to quantify them? Surely research impact must be one of them, but what about pedagogical contributions and departmental citizenship? These latter contributions often helped to save a department from being shut down during those dark ages of statistics. From the perspective of a departmental history, should it be more fair to feature an unsung hero, who helped many students to complete their degrees, more than someone who was busy generating honorary degrees for him/herself?

These were just a few of the many issues that the chapter authors of the volume “Strength in Numbers: The Rising of Academic Statistics Departments in the U.S.” (Springer, 2013) had to contemplate. I often tell my students that the hardest aspect of statistics is that there is no single correct answer, but many bad and ugly ones, though as a statistician I am at least guided by a few reasonable principles. In contrast, I find myself adrift when it comes to deciding how to represent history. The substantial variability in the other 38 chapters seems to indicate that I am not alone.

Perhaps “fairness” is simply a wrong premise to start with for history, but it is hard to ignore because this volume will become “data” for future historians, professional or otherwise. Or perhaps the uncoordinated and idiosyncratic choices at the chapter level are in fact helping to ensure the representativeness at the volume level, because a fair history is less likely to be captured by a stylized “his-story” or “her-story” than by many freestyle “their-stories.” Wait! Uncontrolled individual biases are actually good for collective unbiasedness? Am I insightful or insane?

Who cares? Well, perhaps the editors of future volumes? There must be many readers (I hope there are!) around the world who are wondering, “Where is my department’s history?”


Leave a comment




Welcome to the IMS Bulletin website! We are developing the way we communicate news and information more effectively with members. The print Bulletin is still with us (free with IMS membership), and still available as a PDF to download, but in addition, we are placing some of the news, columns and articles on this blog site, which will allow you the opportunity to interact more. We are always keen to hear from IMS members, and encourage you to write articles and reports that other IMS members would find interesting. Contact the IMS Bulletin at bulletin@imstat.org

What is “Open Forum”?

In the Open Forum, any IMS member can propose a topic for discussion. Email your subject and an opening paragraph (to bulletin@imstat.org) and we'll post it to start off the discussion. Other readers can join in the debate by commenting on the post. Search other Open Forum posts by using the Open Forum category link below. Start a discussion today!

About IMS

The Institute of Mathematical Statistics is an international scholarly society devoted to the development and dissemination of the theory and applications of statistics and probability. We have about 4,500 members around the world. Visit IMS at http://imstat.org
Latest Issue