The Quantitative Peace

Educational Attainment and Race in Alabama

A part of my responsibilities as a post-doc here at Alabama entails helping faculty with seeking out external funding for projects. One of these projects has involved developing a program to promote educational outreach throughout the state. In the course of this work I've been digging through some data on educational attainment rates in Alabama. In terms of learning about the political and economic characteristics of my new state, this has been an informative exercise. I also think it's helpful in illustrating the benefits of digging through your data, and helps to shed some light on the benefits of considering the distribution of your data more carefully, and so I thought I'd write a quick post on the subject. For the purposes of this post I'll be drawing on data collected from the American Community Survey's 2006–2010 five year population estimates. 

The mean percentage of Alabama's population holding a bachelor's degree or higher for this time period is 21.7%.  Compare this figure with that of my home state—New York's overall rate is 32.1%. This is a fairly large disparity, but further disparities emerge when we begin to disaggregate the data by race. The graph below shows the mean value of this same variable across the black and white populations of both states. 

From this we can see that the total values mask some important differences between groups within each state. In both cases the college attainment rate of the white population is slightly higher than the overall mean value. Alternatively, the attainment for the black population is significantly lower than the overall mean value in each state. Even New York, which on its face appears to have a significantly higher overall college attainment rate, is not much different from Alabama when we compare black college attainment rates with Alabama's values. 

Even when disaggregating the data in this way, we often still lean on the mean value pretty heavily. The distribution of the data around the mean can be incredibly informative as well, but we don't often pay as much attention to this as we should. Below I present a quick illustrative example of why this can matter.

Looking at Alabama's data again, I have drawn on county-level college attainment data for both groups.

Looking back at the first graph we can see that the statewide mean white college attainment rate was about 24%, and the statewide black attainment rate is roughly 14%. Looking at the county-level distributions can help us to get a better grasp on educational attainment rates throughout the state. In the case of white attainment, we can see that the distribution is slightly skewed towards the low end, but the lowest value in the distribution is around 8–9%. Alternatively, the distribution of black college attainment rates at the county level is more heavily skewed, and the lowest value in the distribution is considerably lower than for the white attainment rates, as shown in the preceding graphs where the X-axes are held constant to help make the figures more comparable. In fact, .30% is the lowest black attainment rate recorded in the data. 

Looking at the distribution of the data in addition to the mean values for these different groups can provide some additional insights into the substantive issues we're interested in. In this case, it's already apparent from considering the mean values that there are disparities in the college attainment rates of the black and white segments of the state's population. However, these values may actually underestimate the magnitude of those disparities. Though the distribution for white attainment rates is still skewed, it's floor is considerably higher than that of black attainment rates. In fact, by my count there were 14 counties in which the recorded black attainment rate was lower than the lowest recorded white attainment rate. The maximum for each group is roughly the same, however. Areas around Tuscaloosa, Birmingham, and Huntsville have fairly high college attainment rates. But these areas are highly unrepresentative when considering the state more broadly. In fact, although the mean statewide black attainment rate is 14%, the majority of counties fall well below this figure. And while most counties also fall below the mean on white attainment rates, there are, comparatively, more counties falling above the mean. All of this suggests that the mean statewide black college attainment figures are more heavily influenced by outliers in the data than are white attainment rates.  

Lastly, looking at these plots reveals another important feature of the distributions—the variance. The variance in white college attainment rates is 66.2. The variance in black college attainment rates is nearly half that, at only 37.5. This tells us that not only are black college attainment rates lower than white attainment rates, but they are more uniformly so. To put it differently, there is greater dispersion in white college attainment rates, while black college attainment rates are more tightly clustered around lower values. This is made apparent by looking at the two preceding graphs. Though both are skewed towards the lower end of the X-axis, white college attainment figures are less tightly clustered here, as I mentioned above.

This is an issue that I'll come back to in the future, but for now it seemed like this was a useful example for illustrating the importance of familiarizing yourself with the data that you're working with, and how relying less on simple mean values can reveal interesting features of the data you're working with. Bear Braumoeller has an interesting Political Analysis piece about examining the variance, which is worth checking out. More substantively, I also think that this exercise helps to more fully draw out the disparities that may exist between social groups. Relying on simple indicators of central tendency can be useful in some contexts when we want to quickly convey bits of information, but these figures can also be misleading. At the very least, they don't always tell us the full story.

 

Political Science and Scientists in the News

Just a quick link to this Atlantic article. It features several works by political scientists on why and when the public is likely to support military action. Complete with graphs and everything. 

 

*Picture taken from the aforementioned and linked Atlantic article. 

Follow-up

Just a quick follow-up to yesterday's post. Chad sent me a link to this paper that was recently posted on AJPS' early view. I've not read the entire article yet, but it seems apropos given the subject matter of yesterday's piece. 

History vs. Political Science? Temporally Constrained Studies and Generalizability.

So this is my first post back from a prolonged break. As I mentioned in a previous—albeit brief—entry, I’ve had a busy but enjoyable summer. I got married, defended my dissertation at the beginning of July, and my wife and I have since relocated to Tuscaloosa, Alabama where I’ve accepted a position as a post-doc. We’re both pretty excited about the move and we’re really enjoying ourselves. Having grown up in the Adirondacks in upstate New York, summers that extend beyond a two month window have a certain appeal. And now that we’re getting settled in down here, I’m slowly getting back into a routine for work. I’ve been sitting on some papers for a while now that I’m finally turning my attention to, and it’s my experience with one of these papers that motivated me to write this particular post.

I’ve been thinking recently about how we view the place that history holds in our discipline. Now, I suspect there is an initial reaction that many people (particularly quantitative political scientists) are likely to have to the word “history,” and I’m guessing that is probably one of derision. It’s not that political scientists necessarily dislike history or historians, but this reaction is conceivable given that these disciplines are marked by some important epistemological differences, and many quantitative political scientists are often taught from the outset to avoid relying on single cases when drawing more generalizable lessons and conclusions (with good reason). When we think of historians, it is often in the context of someone who provides an excruciating level of detail about one particular event, and then tries to explain its origins and broader implications/lessons. My issue is not in fact with this particular point—as social scientists, we want to be able to assess just how generalizable the relationships that we’re looking at really are. We are typically interested in exploring systematic relationships, trends, and patterns that hold over broader periods of time and apply to a wide range of actors than are many historical studies. So to be clear from the outset, I am not advocating that quantitative political science should attempt to emulate the professional approach or methods that many historians employ.

My issue, really, stems more from the assertion by many quantitative political scientists that a study that is time bound in some way is “history”. This does not apply to all political scientists, but it’s been my experience that reviewers will often look at the temporal range of a study and dismiss its if it is explicitly bound to a particular period of time that does not overlap with the present (allowing a few years of slack on just what constitutes the “present”, that is). Indeed, some seem to be under the impression that it’s not even political science if the gap between the time period covered by a study and the present is sufficiently large. For example, let’s say I’m interested in understanding the domestic determinants of pre-hegemonic US foreign policy behavior. In this particular instance, there are theoretical reasons to suppose that the behavior of the US could be different in this time period as compared to later periods. Let’s further suppose that I have theoretical expectations regarding the relationship between X and Y, and I write a paper on this relationship for the period between 1900 and 1945. I would not at all be surprised to receive reviews rejecting the paper at least in part because the reviewer(s) questioned the study’s broader relevance, given that it’s bound to a 46-year time period.  

I think these sorts of reactions expose some problems and some important assumptions that we often make as political scientists. First, not every study necessarily needs to neatly map onto the present time period—or any other arbitrarily chosen time period. There is nothing inherently unscientific about the notion that certain relationships or phenomena can only be found in a particular temporal context. I’m fairly certain people have not abandoned the study of dinosaurs simply because we no longer see T-Rex roaming around the countryside. And our field is rife with examples of research, the temporal context of which is fundamentally limited in some way. Scholars of American politics provide perhaps the clearest example of an entire sub-field that cannot be held to apply to a period extending back beyond 1789. The sub-field of international relations is similarly dependent in many ways upon the existence of the modern nation-state, which we typically trace back to 1648, and some of the most widely used only go back to 1815. Do we consider our endeavors in these areas “history” because our studies are bound to these time periods?

Also on this point, there seems to be a double standard when we consider the broader implications of our research. Taking my example from earlier, reviewers will commonly ask how the study of US foreign policy between 1900 and 1945 is relevant for today, but rarely do we consider how a study of US foreign policy from 1945–2013 informs our understanding of the 1900–1945 period. It strikes me that this is (1) perpetually moving the goalpost, and (2) that it may be the wrong goalpost. This may seem like an odd point, but what it “relevant” by current standards is something that obviously changes on a day-to-day basis, and it’s a standard that says nothing about the quality of a paper as a piece of social science, or whether or not a paper helps us to understand a particular question about a given set of relationships. Really, this only represents our own innate temporal biases, but it says nothing about how scientific a piece of research is.  If the goal is truly generalizable theoretical and empirical knowledge, in a temporal sense, then this kind of consideration should apply just as much as thinking about how a study informs our understanding of the present. Conducting a study with the purpose of expanding our knowledge of systematic relationships between societal actors is not synonymous with expanding our knowledge of systematic relationships between societal actors for the purposes of informing our understanding of the present.

And it’s not as though we don’t attempt to deal with “unique” time periods and cases in our current research. However, it’s often the case that the manner in which we deal with these cases is fairly crude. For example, we might include a dummy variable in a model to control for a time period (or characteristics of a time period) that we believe to be unique in some way. Bipolarity during the Cold War, for example. Similarly, we might include a dummy variable in our model to control for states that we believe to be unique—depending on our topic, it might be a state like Israel, Egypt, the US, or maybe a group of states like the “Great Powers”. Sometimes we might also include an interaction term to account for how the effect of one variable might be conditional upon another. 

But these approaches are not always appropriate methods for dealing with the questions that we want to answer. Dummying out a particular time period is only going to tell us whether or not a given time period or group has a higher or lower intercept than the alternative time period or group. This approach is also often atheoretical. For example, we might have a belief that a time period is somehow different, but cannot fully articulate why or how. Similarly, interaction terms with a variable capturing a particular time period, for example, are implicitly suggesting that a given relationship is time-bound in some way. However, these approaches don’t allow us to examine whether or not the remaining variables in our models also have different effects in the context of a particular time period. For example, maybe we’re interested in whether or not both regime type and economic interests have a different effect on conflict propensity during the Cold War as compared to after. This, then, would suggest that maybe splitting our sample into two time periods is the more appropriate means of addressing our question. In fact, the notion that we would dummy out a particular time period because we suspect that it’s “different” in some way, but don’t exactly know how, is exactly the reason why we would want to conduct a study that is temporally bound in the first place.

This points to another issue. I think these biases are somewhat rooted in, and reinforced by, our relatively limited access to “good” data. Almost every journal article contains passages wherein the authors attempt to assert the broader relevance of their work. In the case of international relations, it is also quite common for these articles to then proceed to test their arguments using data that is only available for the post-World War II period. Sometimes, these tests will use data for a single country—often the US. Yet the arguments the authors make are often asserted as applicable to a broader set of countries than just the US, and rarely do such papers even address their own temporal limitations. We implicitly accept the generalizability of papers in which the tests of broader theoretical arguments rely on data from an incredibly narrow and often unrepresentative set of states, but push back when a study openly acknowledges its more narrow temporal confines. Why should we automatically assume that such studies inform our understanding of international relations and state behavior in the 1800s?

This is understandable. Particularly in the field of international relations, the availability of data is exponentially greater in the post-World War II period than before. Commonly used indicators like GDP and trade either don’t exist, are often missing, or are highly inaccurate for earlier time periods. Accordingly, many of our studies focus on this 50–60 year time period—not because it somehow matters more, or because we are interested only in this particular time period, but because this is the period for which we have access to relatively abundant data sources. But even in the post-World War II time period some of the data we use can still be of questionable reliability. Accordingly, when we see an article that focuses on a much earlier time period it sticks out like a sore thumb, and reviewers will often proceed to subject that paper to a different standard than other papers—a standard that really has nothing to do with the execution of the paper or the soundness of its argument.

This knee-jerk reaction against studies that are temporally bound in some way can also have deleterious consequences for our ability to understand the world. Finding that a particular relationship between two variables holds only for a given time period can reveal new and interesting questions. For example, we have evidence that Republicans and Democrats have switched their positions on military spending over the course of the Cold War. If we were looking for a relationship between Republicans and higher military spending over the entirety of this time period, we might erroneously conclude that there is no relationship. Alternatively, the finding that this relationship is temporally bound in some way raises new questions: Why did they switch? What caused the switch? Etc.

If our goal is to continuously develop and refine our understanding of how the world works then we must think carefully about the standards we set. If that standard is that we must only examine relationships that hold for centuries at a time, then we are imposing some very serious limitations on ourselves as researchers. These kinds of “big” systematic relationships are clearly important, but the march of scientific progress is not marked exclusively in these terms. 

Tom Coburn's Grudge, Part II

Via Jennifer Diascro's Twitter feed: The Economist has an article up addressing the specific targeting of Political Science Research by Senator Coburn. The article addresses a point that I've brought up a couple of times in the past (see here for example). Specifically, why only political science?

The Economist:

The real debate seems to be over what is of value. For many Republicans the answer is nothing having to do with political science. Yet it is hard to see how that discipline is any more frivolous than—or even all that different from—economics, sociology or anthropology.Mr Coburn's measure wouldn’t touch NSF funding for other social sciences. But this is the fourth time since 2006 a Republican has singled out political science for cuts.

If we look closer at those efforts, the picture becomes a bit clearer. Mr Coburn, for example, has particularly objected to studies on American attitudes to the filibuster, at a time when Republicans are using the filibuster routinely. Mr Flake had likewise complained about research on climate-change negotiations. When Mr Coburn first proposed cutting off funding in 2009, he complained that NSF money had gone to fund The Human Rights Data Project, which, he noted, had “concluded that the United States has been ‘increasingly willing to torture enemy combatants and imprison suspected terrorists,’ leading to a worldwide increase in ‘human rights violations’ as others followed-suit.” Mr Coburn even groused that Paul Krugman had received NSF grants more than 15 years earlier. Presumably that was because Mr Krugman went on to become a liberal columnist and not because the research won him a Nobel prize.

In all of these cases, the research risked calling into question the wisdom of policies supported by the Republican Party. In none of the cases did Republicans argue that the studies were flawed. They appeared to simply object to financing research that might contradict their point of view.

I would add that my impression is that political science research is probably the one field that is most likely to draw upon and combine insights from all of these other fields. Perhaps I'm mischaracterizing the work of scholars from these other disciplines, but if this view is accurate then it further hurts the credibility of Coburn's claims to be going after wasteful spending. If the idea is to get as much bang for our buck as taxpayers, political science seems like exactly the right field to invest in. But does this matter? Probably not. Most people just don't know or care enough about this to really see the problem, and for these people Coburn's credibility is really a non-issue, so he can still claim to be simply fighting the good fight against bloated government budgets. 

Finally, I'd urge all of my Americanist and comparativist colleagues to start building up the national security implications of their work. After all, we must remain ever vigilant to safeguard our democratic institutions from infiltration by the communist scourge. Maybe ANES can just add a question asking "Are you a communist spy?" or "Are you a terrorist?" and Coburn will be satisfied. 

Updates

Just a couple of links, comments, etc.

1. Six seismologists in Italy have recently been convicted and sentenced to six years in prison (each) for issuing a "falsely reassuring statement" prior to the 2009 earthquake in L'Aquila, Italy (BBC Article). The seismologists have been charged with manslaughter over the deaths of over 300 people in the quake, and have also been barred from holding public office ever again in the future. Yesterday's BBC Global News Podcast (Monday, October 22 AM) has some additional details on the situation behind this particular event—apparently there were a series of smaller quakes in the preceding month, but it sounds like such clusters of quakes are pretty common. Some commentators interviewed by the BBC have suggested that this could pave the way for politicians in Italy and other countries to impose new laws criminalizing what I can only label as "bad advice" from scientists, thereby making scientists less likely to offer advice to policymakers. I can't really speak to how likely I think this is, though my initial thought it "not very." Still, I would not have thought it very likely for six seismologists to be sentenced to prison terms under such circumstances in an advanced industrial democracy either. If such laws did become more prevalent, however, I guess I'd be more worried that one potential implication is that failure to come forward to begin with would suggest culpability as well. Damned if you do, damned if you don't?
2. Dan Nexon has recently announced a new blogging reception at ISA's 2013 meeting in San Francisco.  It looks as though nominations for the various award categories are open in the comments thread. Some great blogs already appear to be listed, like Phil Arena's and IPE@UNC. If you somehow read this blog, but not any of these others, you should probably go check them out.
3. The presidential foreign policy debate was last night, and though I don't have a lot to say at this point I do have a couple of brief thoughts. First, Romney definitely seemed more tentative last night than in either of the previous debates. As many others have noted, it seems like this is partially because there's just not a lot of daylight between Romney and Obama on many of these issues. This could also be the result of Obama simply having a greater edge on this subject area because of the job he's held for the past four years, combined with the fact that he's proved far less a lefty on foreign policy than many on the left would have liked, and many on the right had feared. Romney clearly tried to steer the conversation away from what I'll say was a more traditional discussion of security issues to the economy. That's fine, so long as he could have framed it in a clear and convincing way. It's hard to argue that economic policy is not intimately tied in with national security. However, I don't really think Romney did that effectively. His reference to Mike Mullen's earlier comments were helpful, but I think it's getting harder for the Republicans to argue that Obama has reduced our military to some impotent shadow of its former self. Obama's comment on bayonets and horses, while cute, does carry some truth.* Further, that Obama highlighted the fact that we spend more than the next ten top spenders combined on our military helped to further discredit the idea that our military is languishing. He could have increased that number quite a bit and only strengthened that position. In short, I think Romney was weaker here not just because Obama had an experiential advantage, but because even on national security the facts just don't play out as the Republicans would like. 

 

*I'm curious as to how much of this total is simply smaller support vessels that we just don't need anymore. The way to support and maintain ships is quite a bit different now than it was in the early 1900s. The elimination of battleships probably cuts down on this as well. The period Romney referenced is often identified with a significant buildup of increasingly large big-gun ships. 

 

 

Military Capabilities

Phil Arena has a very cool post up wherein he explores a potential alternative to the ubiquitous CINC score for measuring military capabilities. This strikes me as something that is long overdue.  And there are a couple of things that I like about the measure just from taking a quick glance at it. First, given that China just launched its first aircraft carrier, which is actually a refurbished Soviet carrier if I recall correctly, I think the notion that China has already surpassed the US in terms of military capabilities is a big pill to swallow. Second, just glancing at his figures for European powers during the late 19th century, Phil's measure looks more noisy, which in truth is kind of what I'd actually expect to see. The CINC measure makes there appear to be a lot more stability than I would expect to come along with a period where many of these European powers are vying for access to colonial possessions, foreign markets, etc. Furthermore, there were definite differences in terms of how each of these countries allocated their military resources. England certainly had the dominant navy during the late 1800s, but countries like France, Germany, and Russia had advantages in terms of their ability to field ground forces that the British just didn't possess.  

As Phil mentions, the measure may need some further development, but I still think this is really neat. 

On the Utility of IR Research

At the Duck of Minerva, Josh Busby has a post on the gap between political science research/IR and the policymaking community. I don't have a whole lot to say about the specific content of Busby's post, aside from the fact that I think this is an interesting an important debate to have. (Also, please note that the links in the quotations were provided in Busby's original post).

Math's PR Problem OR Changing our Educational System

I was listening to the BBC's Global News Podast this morning and found an interesting segment on the utility of mathematical modeling. Professor John Adam's book (which can be found here) looks at how one can model various aspects of urban life—pollution, traffic congestion, number of dentists offices, etc. I've not read the book, but Adam's message in his interview was something that I can sympathize with. In particular, Adam talks about the degree to which the educational process divorces math from any kind of applied usage. Learning math in this way, he goes on, is partially responsible for why more children/people are not as interested in mathematics as they might otherwise be. 

Sure, there are the ubiquitous "if a train leaves city A at rate X, and a train leaves city B at rate Z, when will their paths cross," kind of examples. But seriously, is that the best we can do? Now, I recognize that I'm far from being the strongest, or the quickest, when it comes to picking up a lot of the methods that I've been taught over the past few years. I, like many people I suppose, was not the biggest fan of math class in high school. Some of this was laziness on my part, but some of it was also the framework in which the subject matter was taught. Even in college it was far from clear that any kind of math beyond balancing a checkbook would play as big a part in my life as it currently does. I did well enough in these classes, but it was not until graduate school that I really had to seriously engage math and statistics.  

As it turns out, I've really enjoyed the quantitative side of my training here at Binghamton, and I'm far more willing to work at developing my understanding and skills now than I was when I was 16. Had I known in high school that I this stuff could be used to study things like war, human rights, etc., I might have cared a bit more. These are things that I've always been interested in. And this is perfectly rational behavior, right? If the expected utility of learning this material is zero, or even negative, then why expend the effort? Where is the payoff? If this is the case, then a major part of the reason that the expected utility of learning this material is so low is because the people that should be able to provide you with the relevant information to update your expectations simply do not do a very good job of that. In fact, I'd say that I didn't really get that at all until grad school. And being able to make that initial connection between math and statistics, and the substantive topics that I care about, has subsequently led me to develop a greater appreciation for the math itself that underlies the models I often use. 

I don't have any answers for how the educational system could be modified to address this issue. I also realize that you would have to find a way to incorporate applied mathematics into most subject fields, as not everyone has the particular set of interests that would make more math in history or government class a sufficient solution to the problem. Still, there must be a better way. With folks like Neil deGrasse Tyson going around trying to promote greater scientific literacy, and the recent attacks on NSF funding for political science, it seems like there is a space for us to modify how we educate our younglings and how science, math, history, etc., can be better integrated in this process. Maybe if people developed a better understanding of science as a process, or math as something with real world and substantive applications, rather than something that existed strictly from 1pm to 1:45pm during 8th period, people would be better able to develop a more robust appreciation for how mathematics and science fit into, and matter for, our daily lives. This is fairly obvious when phrased like that, but the fact is that it's not really what we do currently, is it?

Am I missing something?

Also—and this is a total aside—Brian Rathbun at the Duck of Minerva has a discussion of how Iron Maiden material can be used to teach political science. This is perhaps a good example of what I was talking about above, so perhaps this is not a total aside, but what we have here is essentially "applied awesomeness."

How Bruce Bueno de Mesquita buys a car

Bruce Bueno de Mesquita explains how to buy a car cheaply--something he mentioned on the Colbert Report this week:

 

 

The Colbert interview can be found here.