Most access-to-medicine initiatives are poorly evaluated but moves are afoot to change that

This was originally published on The Lancet Global Health Blog on August 11, 2017.

Two decades ago the World Health Organization and health activists were pressuring global pharmaceutical companies to launch more “access-to-medicine” (AtM) initiatives in low- and middle-income countries. The good news: That has started to happen. The bad news: Startlingly few of these initiatives have any idea what kind of impact they are making.

Those are some of the conclusions of a new study, published in Health Affairs in April. A team of researchers associated with the Boston University Department of Global Health discovered that the number of AtM initiatives from 21 companies had grown from 17 in 2000 to 102 in 2015 but they found published evaluations for only seven of them.

From those seven evaluations, the researchers found 47 articles that met their inclusion criteria for evidence, and all of them were published in peer-reviewed journals. They determined that 62 percent of these were low quality, 32 percent were very low quality and 6 percent were moderate quality. None of them were rated high quality.

The bottom line of the study: “Overall, our findings suggest that current efforts to evaluate the impact of industry-led access-to-medicines initiatives are inadequate.”

The researchers found “clear evidence” that pharmaceutical companies had responded to calls that they increase their commitment to improving access to medicines. But they also found that “the processes that companies use to determine initiative strategies and set priorities are not transparent, which raises questions about whether the resources involved are being allocated in line with the countries’ national priorities.”

But Boston University is not just identifying a problem, it is also leading the way for a possible new model to address these concerns by providing a measurable and transparent way of monitoring and evaluating an AtM initiative in Kenya.

That initiative, called Novartis Access, seeks to provide 15 medicines to address breast cancer, cardiovascular disease, type 2 diabetes and respiratory illnesses at a cost of USD $1 per treatment per month. The project started in Kenya in 2015 and plans to expand to 30 countries in coming years. Novartis Access hopes to reach 20 million patients per year by 2020.

In an effort to avoid some of the mistakes described in the Health Affairs article, Novartis Access – from the beginning – sought out the help of Boston University to evaluate its impact and the prices of these medicines at health facilities and households in Kenya in a completely objective and transparent fashion. Novartis has no control over the evaluation process and Boston University publishes protocols, agreements and all the results (positive or negative) on this website for all to see.

Why evaluate programs like Novartis Access? In addition to satisfying the need of generating evidence of whether such programs are having the intended impact, there is a need to know who is using these medicines.

“One things we know about programs offering low-priced products – not just medicines but also food or fuel – is that it’s often not the poorest people who benefit, but the middle-income groups who know how to access the medicines and the appropriate treatment,” writes Richard Laing, a professor at Boston University who is involved in the evaluation. “For that reason, we’re very keen to find out what happens at the household level, particularly to the poorest families.”

Laing said that what makes this evaluation different from previous efforts to measure the impact of AtM programs is that past efforts have almost always taken place after the fact: “The problem with these evaluations is that you don’t know what the situation was like before, and there is often no control group, so the evaluations have tended to be weak even when the best interventions have take place.”

Dr. Harald Nusser, head of Novartis Access, is excited about the evaluation and its potential to serve a model. “Healthcare companies, including my own, have had a long experience in measuring inputs (how many dollars we invest) and outputs (how many patients we reach, how many doctors we train),” he writes. “But we have not demonstrated the same agility in measuring outcomes or impact when this is the most meaningful way to evaluate public health improvement.”

"Probably the thing I am most excited about is the potential of this evaluation methodology to be used by other pharmaceutical companies to measure their access-to-medicine programs," said Nusser. "Since Boston University is posting all of the protocols and methodologies on their website for all to see, anyone can easily adapt the model to their own needs."

Indeed, Laing said that other companies are already coming to Boston University saying they would like to do a similar study. “By agreeing to this open approach, Novartis has set the standard for others to follow,” said Laing.

The Access to Medicine Foundation, funded by the Bill & Melinda Gates Foundation and the UK and Dutch governments, also helps to keep AtM initiatives honest by compiling an index of the progress made by pharmaceutical companies to make their medicines available. Every two years since 2008, the Foundation has published the Access to Medicine Index. In 2016, Glaxo Smith Kline, Johnson & Johnson and Novartis were named as the leading companies.

The Health Affairs article opines that the index “appears to have created an incentive for expanding AtM initiatives, as well as for more transparent reporting.” But it also says that the index’s ranking methodology does not include a rigorous review of the evidence and should be improved to hold companies more accountable.