Becker's Hospital Review

34 34 CEO/STRATEGY The Man Behind the Hospital Rankings: Reflections From Outgoing U.S. News Editor Avery Comarow By Megan Knowles S ince U.S. News & World Report first published Best Hospitals in 1990, the annual series has become the most publicized and coveted ranking among healthcare organizations. For this we have Avery Comarow, editor and director of Best Hospitals, to thank. Avery Comarow, director of U.S. News & World Report's Best Hospitals and other healthcare rankings, retired Dec. 27 aer 31 years with the organization. For a leader in the methodical world of rank- ings, Mr. Comarow's career path to healthcare journalism was hardly calculated. He says he "stumbled" into journalism, although he paid his undergraduate tuition at the Universi- ty of Maryland as a part-time proofreader for a nearby news production plant of e Wall Street Journal. He attended graduate school at Indiana University to study and teach folklore before changing paths and working at a Bloomington, Ind.-based daily newspaper and then a larger daily in Middle- town, N.Y. From there, he gained experience in consumer-driven reporting as a writer for Money magazine and Washington editor for Consumer Reports before landing an edi- tor position at U.S. News. Mr. Comarow, who ran the "News You Can Use" column of U.S. News, draed the first steps to develop the Best Hospitals rankings aer Mortimer Zuckerman, owner of U.S. News, approached him with the idea in 1987. However, Mr. Comarow worried it was im- possible to rank hospitals — at the time, there was no available data to evaluate them. To start gathering data, Elizabeth Gross, the magazine's then director of research, helped Mr. Comarow partner with a research team at Georgetown University to develop a plan for surveying a small number of physi- cians. e initial survey gathered opinions from 480 physicians — 40 physicians across 12 specialties. Researchers asked the phy- sicians to name the hospitals they would choose to send patients with challenging medical conditions. e survey was based strictly on the physi- cian's perception of a hospital's reputation. "Whatever the physicians thought, we used that," Mr. Comarow said. "I refused to use the word 'rankings' on the cover because statisti- cally there was no way we could take the small number of responses from the 480 physicians and characterize them as robust rankings." To build a more credible system, Ms. Gross then connected Mr. Comarow with the Na- tional Opinion Research Center at the Uni- versity of Chicago. From 1990-1992, U.S. News published rankings driven by physician surveys as the NORC worked to develop the model the magazine uses to organize rank- ings today — the Donabedian paradigm. e move proved to be critical in evolving the list from a limited, reputation-based tool to a credible, data-driven resource. e Donabedian model — a three-part framework to evaluate health services and healthcare quality — establishes a hierarchy for rankings based on a health system's struc- ture, process and outcomes. e framework, which U.S. News first used in 1993, was based on data from physicians and nurses who par- ticipated in focus groups and ranked the attri- butes of hospitals they found most important. Structure analyzes aspects such as the hospi- tal's technology or number of staff, process examines how few mistakes are made during care and outcomes assess whether patients live or die under the hospital's care. But a model informed by focus groups presented challenges — Mr. Comarow said it was dif- ficult to determine if physicians and nurses accounted for how well patients were taken care of, not just whether they lived or died. All while the NORC conducted analyses and provided data, Mr. Comarow worked as the one-man team at U.S. News to complete the published lists. Nonetheless, he is sure to em- phasize the team effort it took to complete the rankings. "If I had tried to do this on my own, I'm not sure this ever would have been pos- sible," he said. U.S. News parted from NORC in 2005 to contract with RTI International in North Carolina. As Best Hospitals continued to evolve, Mr. Comarow was vocal in emphasizing the im- portance of transparency to both consum- ers and hospitals, which guided his role at the helm of an increasing number of U.S. News healthcare rankings. "We have to treat consumers as if they are as smart as we are but don't happen to know as much as we do," Mr. Comarow said. "We give them information and help them understand what it's about. Six different people may make six different choices, but that's what you want. You don't want to make the choices for them." In 2006, Mr. Comarow proposed a session for quality analysts and clinical executives to give U.S. News feedback on its hospital rank- ings. "We invited professionals from 40 to 50 hospitals to let them throw rocks at us for a whole day, and it was amazing," Mr. Coma- row said. If U.S. News hadn't been open to receiving constructive criticism from hospitals, "we would have lost out on input from the people who are in the trenches," Mr. Comarow said. "ey know much more than we do and it would have been idiotic to pretend we were the authority." To keep the consumer's interests and needs top of mind, Mr. Comarow had to push back on data and make it as accessible as possible. "My role was to be an informed medical re- porter who tries to wear the mask of a reason- ably smart but naïve consumer," he said. "Our contractors did all the behind-the-scenes work, but I would push NORC and RTI to come up with data that I could understand." Mr. Comarow remained the only member of the hospital rankings team until current health analysis editor Ben Harder joined U.S. News in 2007. Mr. Harder has since grown the health analysis team to five people. "[Mr. Harder] has been running the show for the last three or four years, but he insisted I keep my title," Mr. Comarow said. Now that Mr. Comarow is retired, he's plan- ning to volunteer at a fire department near his home in Maryland, where he will help with administrative work and recently participated as a simulated patient in an emergency call. "I'm not going to be an EMT, I'm 72 years old," he said. "But I know how to write and I know something about putting numbers together." And the numbers Mr. Comarow put to- gether in the decades he spent with U.S. News endure as the true purpose of the Best Hospitals rankings. "ese rankings really can matter in the most fundamental way — whether patients live or die," Mr. Comarow said. "Whatever harm we might have helped people avoid, that's the real legacy of these rankings." n

• Cover