Rob Pitingolo · Omitted Variable Bias

February 18th, 2010

Another day, another Forbes list goes and kicks Cleveland square in the chin. This time we’re the “most miserable city” (unless you’re a single, of course, in which case, don’t worry because you’re living in one of the best cities in America, according to Forbes). Unlike the winter weather list I criticized last week, the problem with Cleveland’s newest “worst” designation has less to do with the micro issue of badly flawed methodologies and more to do with the macro approach that magazines like Forbes take when evaluating and comparing cities.

When Forbes editors sit around a conference table in their Manhattan headquarters (I’m assuming this is how it happens, who knows what actually goes down), they start with a conclusion and work backwards. For instance, they want to determine the worst weather cities, so they ask, “what makes weather bad?” Somebody shouts “snow” another “cold”. Similarly, they might ask, “what makes for a good singles city?” They brainstorm variables like: cheap rent, cheap beer and the number of profiles on Match.com. What they’re failing to ask is: what makes for good weather or what makes for a bad singles city and then factoring those variables into the analysis.

Think of it like this… when people have a tough decision to make in life, many will put together a pros and cons list. The idea behind this exercise is that it forces you to explore the entirety of an issue instead of focusing only on one side and making a bad decision – to ensure that no important variables are omitted.

What Forbes effectively does is create lists that are exclusively pro or exclusively con. Thinking about cities in these terms doesn’t make any sense.

Cleveland ranks as America’s most miserable city because the data conforms to the variables that Forbes thinks makes a city a miserable place. In fact, when you read Forbes’s “happiest cities in the world” list (don’t bother though, none of Forbes’s lists are worth your time), you notice that they analyze an entirely different set of variables. When we compare cities, we need to compare both the good and the bad; otherwise we draw conclusions that merely confirm pre-determined beliefs.

Last 5 posts by Rob Pitingolo

Random Posts

Category: BFD
1 Comment »

One Response to “Omitted Variable Bias”

  1. Ed Morrison Says:
    February 19th, 2010 at 10:50 pm

    Forbes is a dying brand.

    The magazine is facing continued circulation declines and layoffs. It’s part of a general downward trend in magazine publishing. In October, Forbes announced layoffs. In 2009, ad revenues were off over 25%. To raise cash, Forbes recently sold their headquarters building to NYU. Forbes operates with a high cost structure in a declining market. Bad position.

    These effete lists represent cheap editorial tricks to generate content. They trigger even cheaper PR in a not so clever effort to boost readership. (That happens when even lazier newspaper reporters and editors rewrite press releases instead of doing serious reporting.)

    It’s hard to explain the poor quality of these list, although Rob has done a good job. Perhaps this can help. Forbes compiles a list of the “America’s Best Colleges”.

    On this list, Duke University ranks 104. Forbes believes that Duke ranks below Emory and Henry College (#102), Transylvania University (#85), George Fox University (#58), and Centre College (#14).

    It’s hard to take Forbes seriously. These lists cheapen the brand.

    Cheap editorial tricks are a sign of the accelerating weakness of the magazine. Forbes is in serious trouble. Like the newspaper business, the Internet has blown apart the business models for magazines.

    Unlike the Economist, the Wall Street Journal, the Financial Times, and Investors Business Daily, Forbes has not yet figured out how to generate subscriber revenue from the web. Big mistake.

    Cleveland was around long before Forbes magazine, and the city, no doubt, will be here long after Forbes magazine collapses. (That will happen when the Forbes family has faced enough pain watching their fortunes dwindle.)