Assessing the Effectiveness of CHCF’s Health Journalism Grant Portfolio

Summary

For over 20 years, the California Health Care Foundation (CHCF) has funded nonprofit journalism to help strengthen the media’s capacity to cover health policy in California. The goal of this investment: to ensure that CHCF’s audiences have access to timely, relevant information about the most pressing issues related to the health care and policy landscape.

In 2019, the foundation engaged the Aspen Planning and Evaluation Program (APEP) at the Aspen Institute to assess its health journalism grantmaking. The assessment aims to help CHCF more clearly articulate the strategic framework and criteria for its journalism body of work; assess the effectiveness of these grants; and inform CHCF’s internal discussions about potential refinements to its journalism grantmaking strategy. The findings and recommendations summarized below draw on 81 interviews with journalists, editors, newsroom leadership, and members of grantees’ target audiences; data on digital and broadcast audiences; and surveys and other documents provided by grantees and CHCF. These findings and recommendations, drawn from the internal report that Aspen prepared for CHCF, are intended to contribute to knowledge sharing and field building among a broader audience of stakeholders in journalism grantmaking.

Grantees’ Impact and Value

CHCF’s grants help support high-quality health journalism. Our findings indicate that the health reporting produced by the five public media outlets, California Healthline, and data fellowship meets a high standard of quality. Diverse stakeholders — including health care providers, advocates, individuals in state and local government, and California Healthline’s media partners — confirmed that the health reporting produced with support from CHCF is valued for its depth, balance, nuance, relevance, and attention to system-level factors. And we observed numerous examples of how journalists and their newsrooms strive to reach, report on, and engage with underserved groups, including populations with low incomes and communities of color.

The health reporting supported through CHCF grants has multiple levels of impact. We observed evidence that grantees’ health reporting has impacts at multiple levels of California’s health care and health policy ecosystem. It informs individual community members, sometimes even spurring them to join efforts to address problems identified in the reporting. It contributes to community-level discussions, including on underreported issues. It influences reporting by other media organizations, who pick up grantees’ and data fellows’ stories, invite the journalists to appear as guests on news programs, or do their own follow-up stories. It informs and helps advance the work of advocates and other stakeholders who seek to influence health policy and systems change. And — in more rare cases — we observed evidence where reporting contributed to specific policy decisions and actions taken by elected officials and state health agencies. Of course, stories rarely have a clear through-line to specific impacts. And in some cases, it’s the combined coverage of multiple outlets that makes a difference. Systemic change entails complex, sometimes slow-moving processes involving actors at multiple levels of the health care and health policy ecosystem. Within that ecosystem and despite its complexity and constraints, these seven grantees are collectively making a significant contribution.

Grantees’ health reporting contributes to CHCF’s broader mission by covering its focal issue areas. Our review of a sample of the portfolio’s health stories, which grantees tagged by topic, confirms that a majority of reporting (74%) supported through CHCF’s grants is focused on topics of core interest to the foundation. Moreover, it appears that this alignment between the reporting and the foundation’s core issue areas is not driven by any perceived influence or pressure exerted on the grantees by CHCF. Rather, the topics that grantees see as relevant for their audiences are largely aligned with CHCF’s core issue areas. This underscores the relevance and value of CHCF’s focal areas as it works to improve health care delivery in California, while also confirming that the health reporting supported through these grants contributes to CHCF’s broader mission.

CHCF’s grants contribute to the broader journalism infrastructure that helps bring high-quality journalism to audiences throughout California. Our findings suggest that CHCF’s support for the seven grantees helps shore up critical infrastructure within a fragile health journalism ecosystem. Public media grantees underscored that CHCF’s grants are crucial for ensuring that these stations have the internal resources they need to produce high-quality health journalism. Absent this support, they may not be able to sustain the quality, depth, and focus on marginalized communities that they currently bring to their health reporting. Our findings suggest that California Healthline plays a complementary role in bolstering other outlets’ capacity to provide high-quality health coverage to their respective audiences across the state. Our interviews with California Healthline’s media partners confirm that they value the opportunity to pick up California Healthline reporting because it enables them to provide important stories to their audiences that the partners lack the internal bandwidth to cover.

We also observed strong evidence that the data fellowship strengthens journalists’ data skills – which in turn has a lasting impact on the focus and quality of their subsequent reporting. The fellowship also enables journalists to produce high-quality reporting projects that they might otherwise lack the resources, time, confidence, and editorial buy-in to pursue. These impacts have ripple effects on newsrooms’ capacity to produce stories of this caliber, enabling fellows to more effectively collaborate with data editors, to share their skills with colleagues, and to take on similarly ambitious data-driven projects.

CHCF’s other areas of work intersect with the journalism portfolio’s goal, further bolstering the foundation’s impact on the quality of health journalism in California. CHCF is able to play two roles in supporting high-quality health journalism: as a grantmaker and as a source for journalists. Many of the journalists we interviewed were familiar with CHCF, and several said that they regard CHCF as a valuable source of expertise, research, and data for their reporting. The foundation’s respect for the editorial firewall is one reason why journalists whose work happens to be supported by CHCF are comfortable turning to the foundation as a source for their reporting.

What Is Needed to Ensure a Robust Health Journalism Ecosystem

Newsrooms need — but often lack — time and multiple forms of expertise to produce high-quality health reporting. We gathered diverse perspectives on what it means and what it takes to produce high-quality health reporting. Hallmarks of high-quality reporting include: accuracy; depth, comprehensiveness, nuance, and context; relevance to target audiences; success in helping audiences understand complex issues; use of a systemic lens to reveal underlying problems and the policy or systems levers available to address them; use of multiple sources of information, including data; and attention to the experiences and perspectives of diverse communities, including underserved or marginalized groups. Key capacities needed to produce this caliber of reporting include: availability of journalists with core health reporting skills (e.g., topical knowledge, data skills); sufficient time to do high-quality reporting; tools that can enhance health reporting, such as Excel or Tableau; availability of journalists from diverse backgrounds who bring relevant cultural experience and expertise; and strategies and resources for engaging diverse communities. Newsrooms face formidable challenges in maintaining these capacities at a time when the beat system (in which reporters are assigned to cover specific issues, sectors, or institutions over time) is eroding and buy-outs, layoffs, budget cuts, and staff turnover are commonplace.

Community engagement can play an important role in deepening the quality of health reporting and enhancing its impact — but it requires internal capacity, resources, and buy-in. Across the grantees, we heard many examples of engagement approaches: call-outs on-air and online to solicit input from audience members, in-person community events, targeted outreach via mailings, and collaboration with trusted local media sources. These kinds of activities can help an outlet elevate voices and stories that are often absent from news coverage and reach people outside the outlet’s typical audience. While we heard much interest in community engagement among newsrooms, some outlets have mixed views about the relevance and value of community engagement. Moreover, there are varying levels of internal capacity to engage in these activities.

The Center for Health Journalism and other organizations that support newsroom capacity are important for bolstering the health journalism ecosystem in California. Interviewees across diverse newsrooms named resources that help support their ability to produce high-quality health journalism, including training or educational opportunities, news collaborations, grants, and organizations that support investigative reporting. The Center for Health Journalism stood out for its involvement in all of these supportive activities, even beyond the data fellowship assessed here. It is outside our scope to examine the center’s role within the broader media landscape, but our findings tentatively suggest that the center is a key actor in efforts to support the health journalism ecosystem in California.

When it comes to philanthropy’s role in supporting newsrooms’ capacity to produce health journalism, the quality of the funder-grantee partnership matters. The Knight Commission on Trust, Media, and Democracy recently called for philanthropy to increase its support for nonprofit journalism. But funders who seek to influence grantee newsrooms’ editorial agendas, or narrowly define how grant money can be used, can jeopardize newsrooms’ reputation for independence and prevent them from covering the topics most relevant to their communities. Our interviews with grantees suggest that CHCF’s respect for the editorial firewall and relatively flexible core support grants help avoid such problematic outcomes.

There is some appetite for improved impact tracking — but systematic data collection requires internal capacity, resources, and buy-in. Grantees vary in their level of interest in and capacity to collect impact data. Impact tracking requires internal resources, as well as buy-in from busy journalists and editors who would be asked to spend time entering data about impact. This can have implications for outlets’ ability to demonstrate to their funders that their reporting makes a difference.

Implications for CHCF’s Health Journalism Strategic Framework

Based on this assessment’s findings, we consider implications for CHCF’s health journalism strategic framework, and discuss a set of recommendations for its future grantmaking strategy. These implications and recommendations — described below — are reflected in the following visual, which shows an updated summary of CHCF’s strategic framework. (An abbreviated version of the visual is shown below; the full visual is available in the report under Document Downloads.)

The assessment confirms key components of CHCF’s strategic framework. Our findings suggest that the key capacities articulated in CHCF’s strategic framework — skilled journalists with data skills and topical expertise; media outlets that produce in-depth, locally relevant health care coverage; and the capacity to distribute high-quality health journalism throughout California — are all indeed very important to achieving a robust health journalism ecosystem. The observed impacts of grantees’ health reporting also suggest support for the foundation’s assumption that a robust health journalism ecosystem is an important part of an overall strategy for delivering meaningful change in California’s health care system. We acknowledge that our findings speak most strongly to the direct and indirect effects of grantees’ health reporting on government processes and actions. We surfaced relatively less evidence of how health reporting contributed to impacts on health industry leaders and their organizations’ practices — an area that CHCF may wish to explore further.

Our findings suggest potential refinements in CHCF’s strategic framework and criteria for its journalism portfolio. As reflected in the updated strategic framework visual above, our findings indicate the important role that community engagement can play in strengthening the quality and relevance of health reporting, reaching audiences who may otherwise be underserved by media, and deepening the impacts of health reporting. We also observed the potential role of partnerships or collaborations among news organizations, which can bolster their capacity to produce a more comprehensive series of stories on a topic and enable a smaller outlet that has stronger connections to an underserved population to deepen the quality of reporting coproduced by a larger partner outlet. A related insight: the value of ethnic media outlets and journalists of color, who can bring experiences, expertise, credibility, and an ability to connect that are vital to reporting effectively on specific communities of color. These insights are reflected in the “Partnerships for Scale” strategy in the updated visual.

Recommendations

Clarify the role of the engaged public in the health journalism strategic framework. CHCF has described decision makers and those who influence them as its priority audiences. The engaged public is understood to be a secondary audience that needs to be reached because of its role in helping to set the agenda of decisionmakers and holding these decision makers accountable. As reflected in the revised framework visual, we encouraged CHCF to clarify the extent to which it is a priority to reach a broader engaged public that reflects the full diversity of the state, potentially through outreach and partnerships with ethnic media outlets, and how that connects to both the portfolio’s objective and the foundation’s broader mission.

Learn more about the ways in which health reporting influences health industry leaders. We gathered relatively few insights into whether and how health reporting influences health industry leaders and private health systems. CHCF may wish to further explore whether the direct and indirect pathways through which media coverage informs or influences industry leaders are different from those for state and local policymakers — and whether different strategies may be needed to reach and influence them.

Maintain support for the data fellowship. Our findings indicate that the data fellowship is an effective program. Much of the data fellowship’s value and impact also appears to be driven by the specific focus on data, not only in terms of the skills that journalists honed, but also in terms of the ripple effects in their newsrooms and the broader impacts of their data reporting projects.

Maintain nonprescriptive core support grants to public media stations. We suggest that CHCF continue its strategy of providing core support for health reporting, without imposing additional “prescriptive” parameters that dictate topics or issues that grantees must cover. Our findings suggest that a large majority of the health reporting produced through the seven grants falls within the foundation’s core issue areas — without any explicit direction from CHCF to cover these issues. A more prescriptive approach may negatively affect the foundation’s relationships with grantee newsrooms — which could, in turn, undermine CHCF’s relationships with journalists and its reputation as a trusted and objective source.

Continue to support — and monitor — the distinct value-add of CHL’s aggregation service and original reporting. Subscribers value CHL’s aggregated list of health care coverage — and at least in some cases, consider it more useful or relevant to their work than similar newsletters. But given that target audiences do have multiple potential sources of aggregated coverage (albeit some behind a paywall), we suggest that CHCF continue to monitor the relative value of this service to ensure it is filling a priority gap in the health journalism ecosystem. Our findings also indicate that CHL’s original reporting is valued for its depth, balance, and relevance among audience members and among media partners. And there is clear evidence that CHL’s original reporting is playing an important role in bolstering the capacity of media partners to provide health care coverage — a gap in the ecosystem that otherwise would likely remain unfilled.

Examine whether and how to support community engagement efforts. Our findings suggest that community engagement is a potentially powerful tool, not only for reaching and impacting new audiences but also for deepening journalists’ ability to report on underserved communities and communities of color. Newsrooms need training or mentorship, as well as consultants or staff whose job is specifically focused on helping to plan and execute community engagement initiatives. CHCF should consider whether there are ways it could support newsrooms’ use of community engagement — perhaps by building on the Center for Health Journalism’s model for supporting community engagement, or by providing funds directly to grantees for community engagement training or staff/consultants. This kind of investment — now reflected in the updated strategic framework — may help strengthen alignment between the health journalism grants and CHCF’s emphasis on listening to underserved communities.

Explore ways to strengthen and elevate the contributions of journalists of color to health reporting in California. Our findings suggest that California’s health journalism ecosystem does not fully leverage the strengths and contributions of journalists of color and ethnic media outlets. One potential way to begin addressing this gap is to help build the health journalism skills of journalists of color who may face especially high barriers to entry for training or professional development opportunities. For example, the Center for Health Journalism’s Michelle Levander floated the idea of providing a basic training course to journalists from small ethnic media outlets who might otherwise lack the experience needed to participate successfully in the data fellowship. Another potential option: support content collaboration efforts, such as those supported by the center and those that CHL is exploring with ethnic media partners and freelancing journalists of color. We encourage CHCF to vet these ideas and solicit others in consultation with ethnic media outlets and other stakeholders like Ethnic Media Services, a California-based organization working to enhance ethnic news outlets’ capacity.

Consider how CHCF measures the success and impact of its health journalism grants. Measuring the impact of health reporting is a challenge — from both a methodological and a capacity standpoint. Key questions to consider as CHCF looks ahead to its ongoing efforts to understand its impact include: What evaluative standards and metrics strike the right balance between meaningful and feasible? What kind of support (e.g., training, staffing, software) might CHCF provide to grantees who are interested in strengthening their impact assessment efforts?

We hope our findings and recommendations make a useful contribution to CHCF’s understanding of the value and impact of its health journalism grants and help inform the foundation’s continued efforts to support a robust health journalism ecosystem and achieve its vision of quality health care for all Californians.