As America considers major healthcare reforms, it may have lessons to learn from Seguro Popular, Mexico’s ambitious plan to improve healthcare for its estimated 50 million uninsured citizens, suggests Ryan Moore, co-author of a study published April 8 in The Lancet, a leading international medical journal.

The study, conducted through a partnership of Mexican health officials and researchers from leading American universities, offers a model U.S. policymakers might use to scientifically explore solutions to America’s own looming healthcare crisis, a proven experimental approach capable of providing objective answers to even the most controversial and politically charged questions.

“If the administration has done arms-length science and has involved third parties, like the researchers who were involved in this study, then the case that the administration can make for continuing these programs is much stronger,” said Moore, an assistant professor of political science in Arts & Sciences at Washington University in St. Louis. “They’re more likely to get at the truth — it’s good politics and it’s good science.”

The article, “Public Policy for the Poor? A Randomized Assessment of the Mexican Universal Health Insurance Program,” details a massive, two-year field experiment designed to evaluate Mexico’s push to bring better healthcare to communities ranging from remote villages to crowded urban areas. The study turned dozens of Mexican communities into real-world laboratories where causal effects of the insurance program could be empirically measured and evaluated at the household level as new services rolled out in phases across seven Mexican states: Guerrero, Jalisco, Estado de Mexico, Morelos, Oaxaca, San Luis Potosi and Sonora.

Moore and colleagues developed the experimental design, wrote public-use software to implement it and then “tied their own hands” by publishing a preliminary study detailing exactly how the experiment and analysis would be carried out — a process designed to insulate findings from after-the-fact political meddling.

Researchers identified 74 matched pairs of communities that shared similar demographic and health conditions, and worked with Mexican officials to conduct household surveys capturing a baseline snapshot of each community’s health status. Then, working independent of the Mexicans, researchers randomly selected one from each matched pair of communities for early introduction of Seguro Popular, establishing a controlled framework in which individual changes in health experiences in one community could be empirically compared to control conditions in the matching community.

“This was the largest randomized health policy evaluation ever undertaken,” Moore said. “We the researchers were involved in experimental design, and in charge of data collection and analysis at the other end. Mexican officials had no control over the results and we had full freedom to publish what we found.”

Residents in test areas were encouraged to enroll in Seguro Popular, and participating Mexican states received funds to upgrade medical facilities and improve access to health services, preventive care and medications. Follow-up surveys show the program is making a difference on its primary objective, documenting a 23 percent reduction in families experiencing catastrophic health expenditures.

“This study shows that social policy targeting can be successful,” Moore said. “If money is put into a program targeting the poor to receive health insurance, and if that program is well structured, then the poor can actually see reductions in the amount they pay out of pocket for health care. That may seem obvious, but it’s not. Designing a program that’s targeted in a certain way may not mean that resources actually reach the people it’s intended to reach.”

In fact, the Lancet study identified areas where Seguro Popular needs improvement, showing it’s been slow in reaching some residents. Surprisingly, researchers found no measurable, first-year effect on medication spending, health outcomes or utilization of health services. The bottom line, Moore said, is that without objective empirical evaluations of new programs, it’s difficult to say whether funds are being spent effectively.

“This example of arms-length field experimentation and policy evaluation demonstrates how social science can contribute to bettering individuals’ lives,” said Moore. “A great deal can be gained when policymakers are willing to let science steer the evaluation process, when they’re willing to subject themselves to the possibility of being wrong. When they do that, not only is better public policy made in the long run, but we have a stronger case to make for successful policies in the short run.”

Moore is confident the Seguro Popular evaluation template could be used to guide healthcare reforms now contemplated by the Obama Administration. He points to the State Children’s Health Insurance Program, known as SCHIP, as an example of legislation that already incorporates incentives for states to experiment with funding and services. Some Medicare reform plans encourage experimentation as a way to answer questions about what works best, both on cost and quality of care.

If America wants to be ready to make large-scale changes in its health system, now is the time for small-scale testing. “If researchers are allowed to select these test areas — using scientifically and statistically valid methods — we’ll be able to use experimental methods to do good science, to cut through the politics and get the answers we need,” Moore said. “We can get at truth using these randomized experiments.”

The Mexican Ministry of Health, the National Institute of Public Health of Mexico, and the Harvard University Institute for Quantitative Social Science provided funding for the study. Gary King, a Harvard professor and director of the Institute, is lead author. Other university co-authors include Emmanuela Gakidou and Nirmala Ravishankar of the University of Washington; Kosuke Imai of Princeton University; and Jason Lakin and Clayton Nall of Harvard University. Mexican collaborators include Manett Vargas and Juan Eugenio Hernández Ávila of the Secretaría de Salud, Mexico City; Martha María Téllez-Rojo and Mauricio Hernández Ávila of the Instituto Nacional de Salud Pública, Cuernavaca, Morelos; and Héctor Hernández Llamas of Conestadistica, Mexico City.