July 15, 2008

Best Places Rankings: Why Stats Don’t Necessarily Reflect Reality

The latest Best Places to Live ranking,  released by CNNMoney.com, is a perfect example of how statistical models often miss the point entirely. No doubt the number one city, Plymouth MN, is a nice place– but I certainly wouldn’t want to live there. In fact, I wouldn’t want to live in any of these supposed top ten cities:

1. Plymouth, MN
2. Fort Collins, CO
3. Naperville, IL
4. Irvine, CA
5. Franklin Township, NJ
6. Norman, OK
7. Round Rock, TX
8. Columbia/Ellicott City, MD
9. Overland Park, KS
10. Fishers, IN

Sure, the named cities probably have low crime, affordable housing, job growth, etc.; but if these places are so great, why are they mostly anonymous little burgs relegated to cornfields in the center of the country? To my mind, the modelers left out several key variables:

• Climate: Ever been to Minnesota in January or July? Having spent the first 20-odd years of life in the Midwest, I recall bone-chilling winters and inhumanely humid summers. Global climate change has pretty much done away with spring and fall in the middle states. I prefer not to live in a place where I’m confined inside for at least half the year.
• Diversity:  Though I haven’t studied the ranking cities’ demographics, I’m betting they’re not real diverse populations. As a member of the global community, I prefer a bit of color in my environs–not to mention some spice in my food. During my latest parental visit (I grew-up somewhere between Chicago and Fishers IN), I was unable to find a single Thai, Indian, or Vegetarian restaurant within a 15-minute drive. (Though I did find a couple of Chinese restaurants.)
• Branding:  Humans are a strange lot, with emotions and desires beyond control of rationality and logic. Part of our humanity is a desire to be relevant, to believe we contribute to the world. For lots of us, our identity is tied-up with where we live. While an association with New York or San Francisco might make me proud, hailing from one of these top ranked cities would prompt feelings of inadequacy and irrelevancy. (This human fraility is why we’re so susceptible to marketing. Who wouldn’t prefer an I-Phone over a Blackberry?)

Such are the short-comings of statistical models: if the input variables are wrong or incomplete, the results are meaningless. This ranking falls into that category.

With my day job being in the modelling biz, I’m well aware of the arguments on the lips of these modelers: “We’re only comparing things like quality of schools, housing prices and job growth. These other things are not our concern.” Maybe, but these other things are a major concern for most folks. If your model really did work, this wouldn’t be the first time most Americans have ever heard of many of these places.

I’m not saying these places are evil or bad, but some of the best places to live? In the entire United States? I’m not buying it. Don’t believe me? Ask the next twenty people you meet where they’d most like to live. I bet not a one will mention Roundrock, TX or Norman, OK . . . regardless of what the numbers say.