Several of our members—among them Dr. Austin Frakt, health services researcher, cofounder of the The Incidental Economist (TIE) and a frequent blogger here —have alerted us to some troubling developments related to data access. Word is that the Centers for Medicare and Medicaid Services (CMS) has been directed by the Substance Abuse and Mental Health Services Administration (SAMHSA) to henceforth exclude identifiable data on claims related to substance use disorder diagnoses and treatments from its Medicare and Medicaid datasets.

Needless to say, this is a huge blow for behavioral health services research and health services research in general, as well as the providers and patients who need evidence on what treatments work best, for whom, and in what circumstances. You can find extensive chronicling of the developments from Austin and University of Michigan health law professor Nicholas Bagley , and more on the dire implications for researchers, providers, and patients via TIE here.

In direct response to Austin’s and Nicholas’s persistent (near daily) coverage of this evolving issue, SAMHSA issued an official response to TIE, explaining its decision to direct CMS to now withhold these previously available, identifiable substance abuse data. The decision is rooted in what SAMHSA views as the letter of federal law (42 USC 290dd-2) and the implementing regulation (42 CFR Part 2 (Part 2), which guarantee the confidentiality of patient records of individuals receiving substance abuse treatment services from federally assisted alcohol and drug abuse programs.

As Nicholas notes in his blog post, it seems that “SAMHSA’s position jibes with the text of the rules but is at odds with the spirit in which they were adopted.”

SAMHSA notes that they are, in consultation with CMS and other HHS agencies, “examining ways in which Part 2 may be updated in accordance with SAMHSA’s statutory authority.” That’s the good news. The bad news is that any change in the regulation will take some time, assuming SAMHSA has the legal authority to do it. If it’s determined that they don’t, new legislation granting such authority would be needed. Either way, we don’t see an easy (or fast) fix to help regain access to these data.

AcademyHealth has alerted our advocacy partners in the behavioral sciences and treatment communities, and we will continue to collaborate with them on a coordinated advocacy strategy. In the meantime, we welcome and need your input!

Please share stories of research findings and evidence-based, improvements in care that would not have been possible, or will not be possible in the future, without these identifiable data at! We will use these examples as we make a compelling case to the administration and other policymakers for these data’s release.





Emily Holubowich, Senior Vice President at CRD Associates, is AcademyHealth’s Washington representative and leads the organization’s advocacy efforts in the nation’s capital.

On December 16, President Obama signed the bipartisan Consolidated and Further Continuing Appropriations Act, 2015, or “CRomnibus”–part continuing resolution (CR), part omnibus legislation. The massive spending bill includes a CR through February 27, 2015 for the Department of Homeland Security and 11 separate spending bills for the remaining months of the fiscal year, including the Departments of Labor, Health and Human Services, Education and Related Agencies Appropriations Act (Labor-HHS).

This typically controversial bill, which includes spending for the National Institutes of Health (NIH), the Agency for Healthcare Research and Quality (AHRQ), and other health agencies, generally holds funding ‘flat’ for public health and health research activities. Specifically, the CRomnibus provides NIH a $150 million increase to chip away at funding lost due to sequestration in 2013. AHRQ’s budget is held flat when taking into account the approximately $100 million automatic transfer of funding from the Patient-Centered Outcomes Research Trust Fund. The National Center for Health Statistics (NCHS) is also held flat at $155 million; NCHS has not recouped its 2014 loss of mandatory Prevention and Public Health Fund dollars that were used to expand several surveys.

Flat funding roughly represents the best we can hope for in this austere fiscal environment. However, if you look closely at the details of the legislation and its accompanying report, the CRomnibus does include important ‘wins’ for the field of health services research–wins that may not have been if not for AcademyHealth:

  • Budget Authority for AHRQ and NCHS. The base budgets of AHRQ and NCHS have historically been funded through what is essentially a “tap” on other health agencies’ budgets, making the agencies vulnerable as the fiscal belt has tightened and as lawmakers have sought to fund other pressing priorities (see our blog about one such threat, here). The CRomnibus, for the first time in decades, funds the base budgets of AHRQ and NCHS with full budget authority. Direct budget authority with funds from the U.S. Treasury—rather than relying on other agencies’ budgets—will provide significant stability for the agencies going forward.Earlier this year AcademyHealth, through the leadership of our Friends of AHRQ and the Friends of NCHS coalitions, initiated the “Mind the Tap” campaign to educate lawmakers and staff about the evaluation tap and the important work funded by it, including AHRQ and NCHS, as well as the Assistant Secretary for Planning and Evaluation or “ASPE” and the Office of the National Coordinator for Health Information Technology, among others.
  • Investigator-Initiated Research. AcademyHealth has long advocated for balance in AHRQ-funded research–both what is funded, and how. At our recommendation, lawmakers have for many years targeted funding for investigator-initiated research (IIR) grants at AHRQ to balance investments in intramural research and contracts. Again this year, the CRomnibus includes targeted funding of $45 million for IIR and notes the importance of true IIR while urging AHRQ to avoid being too prescriptive in awarding these funds:

    “Investigator-Initiated Research should not be targeted to any specific area of health services research in order to generate the best unsolicited ideas from the research community about a wide variety of topics.”

    The bill’s report language continues:

    “For this reason, the agreement rejects the administration’s request to target $15 million of the investigator-initiated grants to health economics. No funds are included for this purpose.”It’s important to note that this sentence does not represent a prohibition on AHRQ-funded health economics research, merely, the targeting of IIR grant funding specifically for health economics. AHRQ may still fund health economics through an open IIR solicitation and/or through other areas of its budget. In this regard, this language maintains the true spirit of IIR—competition in the free market of ideas.

AcademyHealth welcomes the CRomnibus for not only what it includes, but what it does not:

  • The CRomnibus includes no bans on patient-centered outcomes research; troubling language that has popped up in draft spending bills of years past.
  • The bill does not prohibit health economics research at NIH, as previous draft spending proposals have. It is important to note that the CRomnibus does not include funding for research “within the [Office of the Director’s] Common Fund specifically related to health care financing reform and insurance incentive activities related to the Affordable Care Act.” This is not an outright prohibition agency-wide; Institutes themselves may continue to fund health economics.
  • The CRomnibus does not include language to restrict behavioral and social sciences at the National Science Foundation. Funding for NSF’s Directorate of Social, Behavioral, and Economics Sciences or “SBE” is an important barometer of policymakers’ attitudes about scientific disciplines other than the basic, clinical, and physical sciences, including health services research (for more on the ongoing “War on Social Science” click here and here).

AcademyHealth worked diligently with our partners in the social and behavioral science community to educate Members of Congress about the value of such research and the federal role in funding it, and the CRomnibus represents the fruits of these efforts.

With the 113th Congress now adjourned, all eyes turn toward January 6 when the new Congress returns and we embark on a new appropriations season in earnest. We will continue our efforts to educate policymakers about the value of health services research and its important place in the federal health research continuum. We were successful in winning the FY 2015 battle and protecting our priorities, but the CRomnibus does include clear signs—particularly around health economics—that we have not yet won the war. We will be calling on you, our members, to help us make an effective case for funding health services research and health data collection.

For more information about funding levels for your specific priorities, please click here for a copy of the legislation, and click here for a copy of the explanatory statement or “report language” (health is in Division G) that includes more specificity about the funding levels.



A new study in the December issue of Health Affairs—led by Medicaid Medical Directors in conjunction with AcademyHealth and with funding support from the Agency for Healthcare Research and Quality, Centers for Medicare and Medicaid Services, and the Health Resources and Services Administration—examines the role state Medicaid programs are playing in reducing the number of early elective deliveries.

In this study, 22 state Medicaid programs with their maternal and child health and vital statistics state colleagues sought to coordinate quality improvement efforts related to early elective deliveries in the Medicaid population. The study finds that almost 9 percent of the 1.8 million-plus Medicaid births each year are early elective deliveries resulting in a higher rate of neonate and NICU admissions or transfers compared to full-term elective deliveries. Furthermore, these deliveries contribute to increased morbidity rates and costs.

While the study also found that early elective delivery rates among Medicaid births dropped between 2007-2012, they still need to be reduced more. Finally, the study identifies policies implemented in states to reduce the rate of these births, including prior authorization (i.e., patients getting permission from their Medicaid plan to have an early, elective birth) and “hard stop” policies (i.e., hospitals prohibiting such procedures), as well as education and feedback efforts targeting patients and physicians.
To access the Health Affairs article:

Additional analysis of the 22-states’ data can be found here:




Over at TIE, I wrote a post a couple months ago highlighting a study showing that breast conservation surgery (BCS) was a better option for many women than more aggressive surgeries like contralateral prophylastic mastectomy (CPM). From the accompanying editorial:

CPM is unlikely to be associated with any significant survival advantage for the general population of patients with unilateral breast cancer. These investigators analyzed survival for nearly 200,000 California Cancer Registry patients with unilateral nonmetastatic breast cancer managed with BCS in 55% of cases, bilateral mastectomy/CPM in 6%, and unilateral mastectomy in 39%. With median follow-up of 89.1 months, 10-year survival for these 3 groups was 83.2%, 81.2%, and 79.9%, respectively. Although the unilateral mastectomy cases experienced a statistically significant survival disadvantage compared with BCS, the absolute difference was less than 4%. These findings support the rationale for encouraging BCS whenever feasible.

In fact, this, and other papers report that BCS has become a “standard of excellence ” in breast cancer care. Unfortunately, trends are going in the wrong direction. “Nationwide Trends in Mastectomy for Early-Stage Breast Cancer“:

IMPORTANCE Accredited breast centers in the United States are measured on performance of breast conservation surgery (BCS) in the majority of women with early-stage breast cancer. Prior research in regional and limited national cohorts suggests a recent shift toward increasing performance of mastectomy in patients eligible for BCS.

OBJECTIVE To examine whether mastectomy rates in patients eligible for BCS are increasing over time nationwide, and are associated with coincident increases in breast reconstruction and bilateral mastectomy for unilateral disease.

DESIGN, SETTING, AND PARTICIPANTS We performed a retrospective cohort study of temporal trends in performance of mastectomy for early-stage breast cancer using multivariable logistic regression modeling to adjust for pertinent covariates and interactions. We studied more than 1.2 million adult women treated at centers accredited by the American Cancer Society and the American College of Surgeons Commission on Cancer from January 1, 1998, to December 31, 2011, using the National Cancer Data Base.

EXPOSURES Year of breast cancer diagnosis.

MAIN OUTCOMES AND MEASURES Proportion of women with early-stage breast cancer who underwent mastectomy. Secondary outcome measures include temporal trends in breast reconstruction and bilateral mastectomy for unilateral disease.

This was a cohort study using the National Cancer Data Base to look at temporal trends in mastectomy, breast reconstruction, and contralateral prophylactic mastectomy. Again, given what we know, we’d hope that mastectomy rates would have decreased in favor of BCS.

That’s not what happened. In fact, the odds of having a mastectomy increased more than one third in the last eight years of the cohort. The rates of increase were highest in women with clinically node-negative disease and in situ disease, women who’d be more likely to benefit from BCS, and not mastectomy.

Breast reconstruction increased from 12% in 1998 to 36% in 2011. The rate of contralateral prophylactic mastectomy increased from just 2% in 1998 to more than 11% in 2011.

Why? It’s not because they were more likely to achieve better outcomes. Research shows that getting more invasive surgery doesn’t appear to improve mortality or survival. It costs more money. It’s a harder recovery. It can lead to more sequelae and problems down the road. There can even be more psychological effects.

One answer can be found in the discussion of the paper (emphasis mine):

Previous work on decision making in patients with early-stage breast cancer demonstrated greater discordance between patient goals and ultimate surgical treatment in women who underwent mastectomy than in those who underwent BCS. Furthermore, less than 50% of women reported being asked by their physicians whether they preferred BCS or mastectomy, and more than 80% of women reported that their physicians made a specific recommendation for either BCS or mastectomy. This suggests that physicians may strongly influence whether a woman with early-stage breast cancer undergoes BCS or mastectomy.

Less than half of women report being presented with the option of BCS. More than 80% report that their doctor basically told them what to do. And research shows us that what women want, and what their treatment provides, often doesn’t match. We can do better than this. We have to improve our communication skills with patients, and make sure that what we do for them aligns with outcomes they value.



{ Comments on this entry are closed }

The following blog post was written by Jia Pu, one of AcademyHealth’s 2014 Delivery System Science Fellows; it is a testimonial about her experience with the Delivery System Science Fellowship (DSSF) and  her host site, the Palo Alto Medical Foundation Research Institute (PAMFRI).

I was thrilled to be selected as an AcademyHealth DSSF Fellow. I know this fellowship will provide the best opportunity for me to continue my research in health care disparities at prestigious health care organizations and to receive unique training in health care system-embedded research. At the end of this very productive fellowship year, I have gained more than I expected, in large part due to the commitment, encouragement, and support of the AcademyHealth DSSF program and my host site, Palo Alto Medical Foundation Research Institute (PAMFRI). Everyone involved in the program is committed to helping Fellows become the next successful generation of scientists in health services research.

My research at PAMFRI focuses on insulin resistance and cardiovascular diseases, including population health outcomes and health care disparities. Working at a health care organization allows me to explore and address my research questions from multiple dimensions.

First, I have access to over 10 years of de-identified electronic health records (EHRs). With this information, I am able to put together the pieces of a disease management puzzle and identify potential opportunities to improve health care and health outcomes. For example, in my recent study of hypertension progression, we examined hypertension and its risk factors based on lab test results from tens of thousands of patients, assessed its association with longitudinal cardiovascular outcomes using diagnosis information from the EHRs, and explored hypertension treatment and medication utilization at ambulatory care settings. This study provides essential information for hypertension management. Without the rich EHR data at PAMFRI, this research would be much harder, if not infeasible, to conduct.

Second, I am excited about the opportunity to work together with patients to explore opportunities to better meet their needs. I have been involved in several patient-centered outcomes studies during my time at PAMFRI. Currently, we are conducting focus groups with patients from different cultural backgrounds in order to better understand their expectations of and experiences with receiving care at PAMF, the large delivery system of which PAMFRI is a part. We are also working together with minority patients to design a website to provide culturally-tailored health information, in particular, those related to lifestyle choices and medication utilization, such as diet and herbal medication. Health care system-embedded research enables and greatly facilitates these patient-centered outcomes research studies with mutual benefits for both patients and health care organizations.

Finally, I am fascinated by the opportunity to learn how health care organizations and local communities can team up to promote community health. For instance, our study team at PAMFRI offered free blood pressure checks at the Asian Indian Community Center during Diwali, a Hindu festival in honor of an ancient Indian holiday. In another instance, free health assessments were provided to taxi drivers along with further health counseling, nutritional advice, and culturally-sensitive recommendations. The concrete influence on our local community well justifies and motivates my day-to-day hard work as a health outcomes researcher, making my career extraordinarily exciting and satisfying.

Besides hands-on experience, I am exposed to the network of all DSSF Fellows. Although we have different skills and expertise, we share the passion to improve the health care delivery system. In addition, I was able to develop my leadership skills by meeting with program leaders, gaining advice from external mentors, and attending and presenting at national conferences. I am very much honored to serve as an AcademyHealth Interest Group Engagement Leader for the Disparities Interest Group, through which I exchange ongoing studies and conduct informal conversations with group members. These social connections inspire and encourage all DSSF Fellows, including me, to overcome challenges in our professional lives.

I believe it is very important for health services researchers to gain additional training and hands-on experience in health care organizations, and the AcademyHealth DSSF offers young scientists a unique opportunity to advance their skills to conduct timely research in a health care delivery system setting. I am very proud to be part of this program and I hope my experience can help shape this program for future Fellows.
Are you interested in working with one of the nation’s leading delivery systems? The 2015 Delivery System Science Fellowship call for applications is now open. Learn more and apply here.


{ Comments on this entry are closed }

When Medicare Advantage plans are paid more, how much greater benefit (or lower cost) is passed on to beneficiaries? A recent paper adds to the growing body of work on this question.

In addition to premiums collected from beneficiaries, Medicare Advantage plans receive taxpayer-financed payments from Medicare to provide at least the basic Medicare benefit to beneficiaries.  Much of the debate around Medicare Advantage is how much more the government pays plans relative to the cost of covering a beneficiary in traditional Medicare. The chart below provides the recent history. There’s no question that Medicare Advantage plans are paid more than average traditional Medicare costs. The question is, what do they do with the extra money?


Because they can provide the basic Medicare benefit at a cost below that which they’re paid, many plans offer additional value to beneficiaries beyond those of basic Medicare. These include reduced cost sharing and extra benefits, like hearing aid or eyeglass coverage.

In a post on The Upshot in July, I summarized the literature on just how much more value beneficiaries receive from Medicare Advantage plans per dollar of additional payment.

[A]t least three studies suggest that for each dollar of these higher payments that plans receive, beneficiaries get only a fraction of a dollar of value. A study by Harvard scholars found that a $1 increase in payment translates to at most 50 cents in additional benefits. Another by researchers from the University of Pennsylvania found that only 20 cents of each additional dollar in plan payment is converted into better coverage. Finally, my own work with my colleagues Steven Pizer of Northeastern University and Roger Feldman of the University of Minnesota found that only 14 cents per dollar of additional payment benefits Medicare Advantage enrollees.

Now there is a forth study in this area. Marika Cabral, Michael Geruso, and Neale Mahoney examined the effects of a change to Medicare Advantage payment rates ushered in by the 2000 Benefits Improvement and Protection Act (BIPA), which raised payments to plans in 72% of the nation’s counties. Using a difference-in-differences approach that compared counties where payments went up to those in which it did not, the authors found that 53 cents of each dollar of payment increase translated into benefits or reduced cost sharing or premiums for Medicare Advantage enrollees.

The authors decomposed this “pass-through” of higher plan payments as follows. For each dollar in additional payments to plans, beneficiary premiums fell by 45 cents and the total actuarial value of benefits, reflected in things like lower copays and extended supplemental benefits, increased by 8 cents. So, in total, beneficiaries were better off by 53 cents for each dollar in additional government payment.

Their study also provides several other useful insights. First, they found limited evidence of favorable selection into plans (enrollment by healthier than average beneficiaries). Second, they found that competition in the Medicare Advantage market affected the extent to which higher payments to plans benefit enrollees. In the most competitive markets 75% of plan payment increases were passed through to beneficiaries. In the least competitive markets, only 10% were.

It’s this final result that I find to be the most interesting contribution to the existing literature. Competition really matters. Beneficiaries in highly competitive Medicare Advantage markets can, in theory, receive close to the full benefit of each additional taxpayer dollar in plan payment. But no markets achieve that, and, on average, markets are quite far from the ideal. Close to half of each dollar in additional payment from the government is retained by plans, typically.

One of the main arguments in favor of Medicare Advantage is that competition provides the greatest value to beneficiaries. Even if one believes that to be the case, according to this study, Medicare Advantage may not achieve levels of competition to come close to fulfilling its potential. Beneficiaries are supposed to be better off under competition. The less competitive the program, the less clear it is that they are.

Austin B. Frakt, PhD, is a health economist with the Department of Veterans Affairs and an associate professor at Boston University’s School of Medicine and School of Public Health. He blogs about health economics and policy at The Incidental Economist and tweets at @afrakt. The views expressed in this post are that of the author and do not necessarily reflect the position of the Department of Veterans Affairs or Boston University.


{ Comments on this entry are closed }

On October 29, AcademyHealth and America’s Essential Hospitals co-hosted an Innovations Summit in Washington, D.C. to highlight how social networks improve population health. The blog below, by AcademyHealth Research Assistant Danielle Robbio, offers a snapshot of the event.

Social networks span beyond the chime of a tweet or “like” of a photo. Our connectedness to those in our community, our friends, family and colleagues—our real-world connections—determine our behaviors and ultimately, our health. The 2014 Innovations Summit began with a presentation by Dr. Nicholas Christakis on his social network research that illustrates the behavioral phenomenon of “social contagion.” Studies show that who you target for an intervention in a network is critical to the success or adoption of the intervention; knowing who to target for systemic change will greatly improve the uptake of the desired change. This work is of particular interest to the health services research and policy community as we look to design interventions that impact a variety of communities in order to improve population health

The illustration social networks’ importance in the wake of a health systems remodel provides much needed insight for those involved in improving population health. However, it also forces us to self-reflect and ask not only who are we targeting for change, but what kind of change are we trying to make.

In addition to the presentation by Dr. Christakis, panelists representing different health systems across the country presented their unique approaches to care and the innovations their organizations are fostering. Although each system is geographically and demographically different, every panelist demonstrated that their purposeful investment in each individual in their social network (via time and non-financial methods) made the difference in health outcomes. For example, in Arizona the Refugee Women’s Health Clinic incorporates cultural competencies into their service delivery. Midwives are trained as cultural health navigators—building trust and appropriately administering care. The ability to connect with the community has allowed the Center to make real changes in the health of the social network they serve.
Over lunch, participants debated key questions related to social networks and health:

  • How do you define population health?
  • What are some challenges to practicing population health?
  • How can social networks support population health activities?
  • What roles can researchers, policymakers, and/or communities do to promote the field of population health?
  • What unique roles do essential hospitals have in providing population health programs and services? How does this role differ from hospitals that don’t have a commitment to serve the underserved?
  • What would you propose as measures or metrics for measuring the success of population health programs?

These questions promoted lively discussion among participants. Specifically, challenges and opportunities were identified.

Challenges to improving population health:

  • Leadership—defining who is responsible for assuring care can be difficult, especially when population health is not uniformly defined.
  • Infrastructure—Data quality and access issues have proven to be roadblocks to improving population health. Harnessing data sources and utilizing them appropriately has the potential to improve research and delivery of services. Additionally, programmatic and physical infrastructure can be problematic for the delivery of services that are relevant to population health e.g. EHR adoption and use; clinic location (urban vs. rural).
  • Sustainability—in order to continue improving population health, we must think beyond grant funding and develop models of sustainability.

Opportunities for action:

  • Trust—understanding our social networks and their priorities is the most important strategy.
  • Contact—deliberately and mindfully designing interventions and strategies to target key members of our social networks will vastly improve our success.
  • Partnership—collaboration with other sectors (i.e., housing; transportation) will allow us to align clinical care and social services. By leveraging other resources and engaging various stakeholders, we can holistically improve population health.

As we move forward with health systems change, we heard clearly that the networks with whom we are working are critical to our strategy and to our success. We also know that the strategy must be inclusive and purposeful. In the words of panelist Gilbert Salinas, “We no longer ask patients what’s the matter with you, but rather what matters to you.”

Let us bear this in mind as we move forward with improving population health.


{ Comments on this entry are closed }

Do NNTs work?

by The Incidental Economist on November 14, 2014 · 1 comment

I’m a big fan of the NNT or “number needed to treat.” But I’d drop my support if it turns out the NNT is not informative or helpful.

The NNT tells you, for a specific therapy and condition, how many patients need to be treated for each additional positive outcome relative to a control group. For instance, if for every 227 people treated for a condition, one additional person is alive after five years compared to a control group, the NNT for this outcome is 227. (Additional description here and, in a video by Aaron, here.)

There is an analogous metric called “number needed to harm” or NNH that measures how many people need to be treated to obtain one additional negative outcome. For example, if for every 97 people treated, one more suffers a stroke within five years, compared to a control group, the NNH for this outcome is 97.

At the website, physicians have amassed a large number of NNTs and NNHs for a wide range of conditions, treatments, and outcomes. These all come from clinical trials. Though they need continuous updating as new evidence comes out, I am not aware of any more comprehensive and accurate collection of such numbers. (More about that site and its creator in this Wired magazine piece.)

The great hope for NNT and NNH is that they motivate more appropriate decision making about which therapies are best for which patients. One patient may be perfectly willing to trade an NNT of 227 for five-year survival for an NNH of 97 for a stroke in that time. Another person might not be. It’s subjective. At least the NNT and NNH let patients make that trade off on a basis of evidence. But there are other metrics of benefit and harm that could be used. Maybe they’re better.

What’s not subjective is how well people comprehend NNTs and NNHs relative to other metrics. Are people able to reason with them rationally, or at least more rationally than with other metrics of treatment effectiveness and harms, like relative and absolute risk reduction? (Relative risk reduction (RRR) is the reduction in risk in the treatment group relative to the control group. Absolute risk reduction (ARR) is the absolute difference in risk. If the risk is 10% in the control group and 5% in the treatment group, the RRR is 50% (=5%/10%), the ARR is 5% (10%-5%), the NNT is 20, or the reciprocal of the ARR.)

Two systematic reviews addressed this question, among others. One is a 2011 Cochrane review by Akl and colleagues. The other, by Zipkin and colleagues, was published in the Annals of Internal Medicine in August of this year. Both reviews found that NNT (and, because they’re basically the same, I would assume NNH) is harder for people to understand — in the sense of successfully using them for probabilistic reasoning — than other metrics of risk reduction (or increase). In addition to comprehension, the reviews also examined perception, persuasiveness, satisfaction, and decision making, none of which I address in this post.

To see more specifically what the reviews’ conclusions mean, I took a closer look at the studies that informed them. The Cochrane review’s conclusion is based on a single study that compared NNT to RRR, Sheridan et al. (2003). The Zipkin et al. review also cited Sheridan et al. (2003) and three other studies. However, two of those three other studies do not assess probabilistic reasoning. The only other one cited that does so is Berry et al. (2006).

Let’s look at the two relevant studies in turn.

Sheridan et al. (2003) was a randomized survey of 350 adults, each presented with ARR, RRR, NNT, or all three for two drug treatments of a hypothetical disease. They were then asked which treatment provided greater benefit and to compute the effect of one treatments for a given baseline risk of disease.

When asked to state which of two treatments provided more benefit, subjects who received the RRR format responded correctly most often (60% correct vs 43% for COMBO, 42% for ARR, and 30% for NNT, P = .001). Most subjects were unable to calculate the effect of drug treatment on the given baseline risk of disease, although subjects receiving the RRR and ARR formats responded correctly more often (21% and 17% compared to 7% for COMBO and 6% for NNT, P = .004).

First of all, it’s pretty clear that adults, at least those in this study, are terrible at these tasks. Second, in the realm of terrible, NNT was the worst. Still, we should not be satisfied with any of these approaches to communicating risk.

Berry et al. (2006) was principally a study of whether providing baseline risk improved people’s reasoning about risk. It does (no surprise). But within the results, some information about NNT (NNH, actually) can be inferred. The study is based on a convenience sample of 268 adult women. Each was randomized to receive ARR, RRR, or NNH information about a harm from second versus third generation oral contraceptives. Each was also randomized to receive baseline risk information (the harm from the second generation pill). Participants were then asked what they think the risk of harm is for each of the pills. The answers are 0.02% and 0.04% for the second and third generation pills, respectively.

First of all, it’s impossible to obtain these answers from ARR, RRR, or NNH without baseline information. I suppose participants might have some baseline in mind, with which they can compute the harm for the third generation pill. But I’m not interested, in this post, in whether providing baseline information is helpful. It seems obvious that it would be, and that’s what the authors found.

So, let’s move right to the comprehension results for the with-baseline sample. Here, those who were provided NNH got closer to the right answers than those who received ARR or RRR information. Still, they were way off. Those receiving NNH information estimated second generation risk of 0.55% and third generation risk of 1.74%, which are 27.5 and 43.5 times larger than the right answers, respectively.

These are terrible! But participants who received ARR and RRR information did far, far worse. So, this is not a study that suggests NNH (or, by extension, NNT) leads to worse comprehension than other measures. It performed better (but still awfully).

So, the evidence base is both thin and inconclusive. There are two relevant studies about NNT comprehension included in two systematic reviews. They point in opposite directions.

My conclusion is that NNTs may, in fact, perform quite well relative to other measures. Or maybe they don’t. We don’t know. They may perform even better for practitioners than for patients, but we don’t know that either. What we can’t say from the evidence, however, is that NNTs are harder for people to understand than other metrics of risk. From two studies with conflicting findings, we just don’t know that.

Austin B. Frakt, PhD, is a health economist with the Department of Veterans Affairs and an associate professor at Boston University’s School of Medicine and School of Public Health. He blogs about health economics and policy at The Incidental Economist and tweets at @afrakt. The views expressed in this post are that of the author and do not necessarily reflect the position of the Department of Veterans Affairs or Boston University.


{ Comments on this entry are closed }

In mid-September, the Patient-Centered Outcomes Research Institute (PCORI) released for public comment a proposal on the process for peer review and public release of its primary research findings.

You may recall that PCORI was established by the Affordable Care Act (ACA) as a congressionally chartered nonprofit “to improve the quality and relevance of evidence available to help patients, caregivers, clinicians, employers, insurers, and policymakers make informed health decisions.”

Operating under its mission and mandate, PCORI created a proposed process for conducting peer review of the primary research it funds and making the findings publicly available within a specific timeframe. As stated in a PCORI press release, “The effort is part of a broader ‘open science’ framework that PCORI is developing to promote transparency and data sharing within the research community and the general public.”

On behalf of its membership and the field of health services research, AcademyHealth provided official comments on the proposed process given the importance of patient-centered outcomes research and the dissemination of its findings to our community. Specifically, AcademyHealth commented on the areas most aligned with its mission and policy priorities and those that best reflect researchers’ interests.

At their most rudimentary level, AcademyHealth’s comments supported the proposed process and offered suggestions for improvement in the registration of PCOR studies; the review, reproducibility, presentation, and reconciliation of research findings; and the ongoing evaluation of the peer-review process.

Read AcademyHealth’s full comment submission here.


{ Comments on this entry are closed }

Every year, I like to write about the Health Care Cost Institute’s annual report on costs and utilization. The 2013 report was just released. There’s a lot of interesting information in it:

This report, 2013 Health Care Costs and Utilization, is the fourth in a series of annual reports by the Health Care Cost Institute (HCCI) on the health care activity of individuals who are younger than age 65 and covered by employer-sponsored health insurance (ESI). The report’s study period (2011 -2013) covers the years after passage of the Affordable Care Act (ACA) and prior to the opening of health insurance exchanges. As in previous years, the report details the levels and changes in per capita expenditures (“spending”), utilization (“use”), and prices of medical and prescription services used by the ESI population. Also, for the first time, it details patterns of spending and service use by age-gender groups of the ESI population.

One of the nice things about the HCCI report is that it tracks spending in the employer-sponsored health insurance market, which covers a majority of people in the United States. Often, databases cover government or public programs, so these data give is a different picture than others might provide.

Overall health care spending only rose 3.9% in 2013. In fact, health care spending since 2010 has grown by 3.9% a year. That’s somewhat remarkable given the grown in spending in the decade before.

Some factors seen in 2011 and 2012 continued in 2013. About 20% of spending was on acute inpatient care, and 28% was on outpatient care.  An additional 34% were for professional services, and 17% were for prescriptions. Outpatient services were the fastest growing sector of spending, but growth slowed last year to 5.2%. Spending on professional services grew only 0.8%, mostly because of increased utilization.

Generic drug use went up once again, by 4.5%. This raised overall generic drug use to 83.3% of prescription filled days.

Inpatient care saw a number of decreases. Admissions, outpatient visits, and outpatient services all went down. The use of brand name drugs used in the inpatient setting also decreased, by more than 15%.

However, some spending decreases were compensated for by increased pricing. For instance, inpatient admissions, outpatient visits, and brand prescription prices all went up, moderating some of the effects of decreased utilization. Admissions went down by 2.3%, but prices for them went up by 5.7%. Outpatient visits went down by 0.8%, but prices went up by 6.4%. And, as I mentioned above, inpatient brand name drug use went down more than 15%, but prices went up by more than 21%.

This means that, once again, the increases we’re seeing in health care spending aren’t driven by using “too much” of it, or even “more” of it. We’re consuming less and less each year. It’s the prices. They go up even faster than the utilization goes down. As long as that happens, we’re going to continue to spend more on health care, even as we become more discriminating consumers of it.

Of course, the first full year of the Affordable Care Act’s real changes were in 2014. This report covers 2013. To see if it had any real effect on health care spending, we’ll need to wait until next year’s report. It’s not quite clear how the ACA will affect spending in the employer-sponsored insurance market, which it doesn’t affect too much, but many are hopeful that we’ll see something. We’ll know a year from now.



{ Comments on this entry are closed }