More Guns Less Crime by John Jr (best free e book reader .txt) 📕

people who get permits. Focusing only on setting fees to recoup the costs of the permitting system will end up being financially short sighted.

How Sensitive Are the Results to DifferentSpecifications?

While I have tried to control for all sorts of factors that might explain changes in crime over time, it is indeed possible to get overzealous and account for too many variables. Including variables that do not really affect crime can actually create problems similar to excluding factors that should be included. Take a simple example of explaining how the stock market, say the Dow Jones industrials, changes over time. Obvious variables to include would be the interest rate and the expected growth in the economy, but many other variables—many of dubious importance— could possibly also be included. The problem arises when such variables are correlated to changes in stock prices merely by chance. An extreme case would be including the prices of various grocery store products. A store might sell thousands of items, and one—say, the price of peanut butter—might happen to be highly correlated with the stock prices over the particular period examined. We know that peanut butter has little to do with explaining overall stock prices, but if it just accidentally happens to move up and down with the movements in the stock market, other variables (like the interest rate) may no longer prove to be statistically significant.

There are ways to protect against this "dubious variable" problem. One is to expand the sample period. If no true causal relationship exists between the two variables, the probability that this coincidence will continue to occur during future years is low. And this is exactly what I have done as more data have become available: first by looking at data through 1992, then extending them to 1994, and now up until 1996. Another approach guarding against the "dubious variable" problem is to replicate the same test in many different places. Again, this is exactly what I have done here: I have studied the impact of right-to-carry laws in different states at different times. As charged by many a critic, it is still conceivable that some other factor just happened to occur also when an individual state passed the law, but the probability of mere coincidence falls as the experiences of more and more states are examined. It is also possible that

EPILOGUE/ 185

adding variables that don't belong can cause you to get a more significant result for other factors than is warranted.

Generally, excluding variables that should be included is a more significant problem than including variables that should not be included, and in general I have tried to err on the side of including whatever possible factors can be included. Indeed, a strong case can be made that one must be careful not to include too many variables like state time trends, which can be endlessly added on and have little theoretical justification. Still, I do not consider any of these variables to be similar to the price of peanut butter at the local grocery store in the previous discussion, but obviously some researchers might believe that some variables should not be included. One way to investigate this issue is to include only those variables that different investigators view as relevant. In the early stages of my research, when I presented my original research as a working paper at seminars, I asked participants for other factors that should be included, and some of their comments were very helpful. I also tried in vain to ask pro-gun-control researchers what variables they wanted me to include in the regressions, but (as discussed in chapter 7) they did not make any suggestions when my initial research was circulated for comments. What comments they made after the publicity broke claimed that I had not controlled for factors that I had indeed accounted for.

Since the original research immediately received a lot of attention, I have let my critics decide for themselves what variables should be included by simply giving them complete access to the data. I know from personal communication that some critics (such as Black and Nagin) did indeed examine numerous different specifications. 21

A more systematic, if time-consuming, approach is to try all possible combinations of these so-called control variables—factors which may be interesting but are included so that we can be sure of the importance of some other "focus" variables. 22 In my regressions to explain crime rates there are at least nine groups of control variables—population density, waiting periods and background checks, penalties for using guns in the commission of a crime, per-capita income, per-capita unemployment insurance payments, per-capita income maintenance payments, retirement payments per person for those over sixty-five, state poverty rate, and state unemployment rate. 23 To run all possible combinations of these nine groups of control variables requires 512 regressions. The regressions for murder rates also require a tenth control variable for the death-penalty execution rate and thus results in 1,024 combinations of control variables. Given the nine different crime categories, this amounts to 5,120 regressions.

This approach is decidedly biased toward not finding a consistent effect of the right-to-carry laws, because it includes many combinations of control variables that no researcher thinks are correct specifications. Indeed, even the strongest, best-accepted empirical relationships usually fail this test. 24 Since different people will have different preferences for what variables should be included, this massive set of results makes sense only if one knows what variables produce what results. If a range of conflicting estimates are then produced, people can judge for themselves what they think the "true" range of the estimates is.

Two sets of variables have been primarily used to test the impact of right-to-carry laws: crime trends before and after the adoption of right-to-carry laws and the percentage of people with permits. Yet another division is possible by focusing on counties with a large number of people to avoid the difficulty that low-population counties