Funder: Kaiser Permanente Community Benefit

Author: Mathematica Policy Research

December 2016 | Technical Assistance Tool


Many assessments of innovations in health delivery, including those of community paramedicine programs, focus on short-term effects using data from time-limited pilot programs. Because they are focused on a short, limited timeframe, these assessments often underestimate programs’ long-term promise. However, assessing both the short- and long-term outlook of a program is likely to give a clearer picture of a program’s value.

With support from the Center for Health Care Strategies, Mathematica Policy Research developed a tool to help the Commonwealth Care Alliance (CCA) assess the “business case” for expanding its Acute Community Care paramedicine program. This tool forecasts costs and savings under different implementation and expansion scenarios, and identifies cost and savings drivers such as patient volume, health service costs, and operating costs, and how these drivers affect financial performance.

Although it is designed to assess CCA’s specific community paramedicine program, the tool provides an example that may be helpful for others who are exploring health delivery innovations, and who may have information on short-term benefits and costs, but require assistance in examining longer-term value. Users can enter inputs into one or more cost drivers to see how savings projections change. This information can inform programmatic decision-making by illustrating various scenarios. To modify this tool or develop one similar to it, consider the following principles:

  • Identify a comparison strategy and analyze service use. As a starting point, the tool should incorporate findings from an analysis of program effectiveness that compares participants to a similar population. It is critical to have a counterfactual (i.e., a comparison group) to understand program effects. Contemporaneous comparison groups are ideal, but program sponsors may also consider looking at intervention participants prior to implementation if multiple years of pre-period data are available.
  • Consider important subpopulations. Program participants will not be homogeneous and may come from important subgroups that require tailored comparison strategies and potentially tailored program cost estimates. The tool should explicitly account for subpopulations, such as groups based on payer or geography, and outline how they matter to the business case.
  • Account for all program costs. Assessing pilot program costs as well as future operating expenses is critical to gauging the potential financial sustainability of a program. Program costs should be documented clearly for all users.
  • Allow for program elements to change over time. Future implementation of a program will be different from its implementation during the pilot phase. As such, any tool used to gauge future program effectiveness should allow users to account for potential changes in program elements such as the number of participants, operating costs, or other primary parameters that drive the business case. For example, if the number of participants increases, operating costs are also likely to increase.
  • Keep it simple. A tool is only useful if it is used. Tools with simple interfaces and easy-to-understand instructions are more likely to be used than complex tools. When developing the tool, obtain feedback from the expected users to make sure the tool is accessible and relevant to the task at hand.