“Ranking studies often attract media and government attention and
are taken seriously, regardless of their designs and limitations,” say
Sandra A. Ham, M.S., and colleagues in the American Journal of Public
Health. “Nevertheless,
controversies exist about whether ratings accurately reflect the ‘livability’ of
cities and the extent to which such reports can be misleading.”
The ideal way to compute these rankings would be to start with environmental
and behavioral measures weighted to account for their roles in producing
health or illness, the researchers say.
But this ideal system doesn’t exist. Everything
from geography to the interpretation of data may serve to undermine the
credibility of
the lists, Ham says.
For instance, health authorities gather some information
at the state level, some only within cities, and some by metropolitan
statistical
area — a
city and its surrounding counties.
“State-level averages may not adequately represent the health situation
in any of the state’s cities,” she says, adding that combining
published reports from various sources only obscures data. Rankings also
oversimplify the complex interactions among behavior, environment, health
conditions and population dynamics.
Exactly when health data was measured or how it is defined
can make a difference too, she says. Data gathered on obesity in the
early 1990s
can’t
be compared to material from the end of the decade when obesity prevalence
shot up. The scientific meaning of “overweight,” “obesity” and “physical
activity” have all changed in recent years, but both old and new
statistical definitions are often lumped together.
Finally, while it’s easy to group the best or worst
two or three cities using these indicators, she says, the scores of the
in-between
places are often statistically indistinguishable from each other, leaving
the
rankings for most towns in a meaningless limbo. Still, there can be some
value in city rankings, despite their flaws.
“Ranking cities can play an important role in raising awareness
of public health issues,” Ham says. “Such data also can be
used to raise awareness among residents … and to hold government
and the private sector accountable for doing what is necessary to keep
residents healthy.”
However, readers ought to keep in mind the methodological limitations
of these rankings, she says.
