Introduction

Public-private partnerships, or PPPs is a term that can be used to describe a broad category of activities and structures involving public and private sectors, defined by the World Health Organization [WHO] broadly as: “wide variety of ventures involving a diversity of arrangements, varying with regard to participants, legal status, governance, management, policy-setting prerogatives” [1]. At minimum, a PPP involves one core public, non-commercial organization and one private or commercial organization that join to share efforts and rewards of the collaboration [2]. It is generally agreed that PPPs have or should have social value, or contribute to improved health of vulnerable populations, generally in terms of increased healthcare system efficiency and access, decreased costs, and increased innovation or creativity [3]. The results of a 2014 systematic review revealed there is a strong focus within the extant PPP literature on efficiencies attributed to graduated models of public healthcare infrastructure and delivery support delegated to the private sector [4]. The most common PPPs involve traditional procurement contracts to support physical infrastructure or building maintenance [5]. However, there is increased interest in initiation of PPPs that establish new PPP goals such as public health intervention scale up and health systems capacity building [6]. The characteristics that drive innovative hybrid PPPs, such as bypassing bureaucratic public contracts and governance, raise crucial questions as to how such PPPs pose both benefits and limits by design. Likewise, they require expanding outcome evaluative parameters to include considerations related to PPP inputs and sustainable development [7, 8].

Metrics that quantify PPP-strategy outcomes beyond finance and health indicators are often neglected in public health research. Using a value-based partnering lens can shift focus towards the quality of relationships and network partner interactions which can enumerate benefits beyond disease transmission trends and viral suppression rates [9]. Reflecting on synergies and compromises made during PPP implementation has the potential to reveal strategic avenues to create and nurture long-term impactful relationships and individual organizational engagement [10]. Retrospective evaluations addressing global health PPP sustainability are rare, yet they provide the ability to explore both expected as well as unintended impacts, which are critically important when exploring lasting impacts of partnerships [11, 12].

Background

The commitments, orchestration, and perseverance of time, resources, and expertise required to achieve the ambitious United Nations Program on HIV/AIDS (UNAIDS) goals and to meet global 90–90–90 targets for epidemic control are highly complex. The United States President’s Emergency Plan for AIDS Relief (PEPFAR) operationalizes the implementation approach guided by the Office of the U.S. Global AIDS Coordinator’s (OGAC’s) priorities that ultimately work towards the UNAIDS 90-90-90 targets [13]. A critical recognition in PEPFAR 3.0 is that public support and resources required for this effort exceed the capacity of any individual governmental organization and/or international partner to bolster system-level capacity to effectively maintain HIV/AIDS disease control and prevention in low-income settings [14]. Therefore, there is an increasing presence of PPP projects dedicated to global health system improvements involving resource sharing from a broad representation of government and private sector organizations [15–17]. Touted as one the many benefits of PPPs is offering latitude to partners on how to operationalize projects, as they differ from traditional service delivery contract models in terms of flexibility and have a less apparent risk and reward structure [5].

The aim of the UNAIDS Scale Up Goal agenda was to diagnose 90% of all HIV-positive persons, provide antiretroviral therapy (ART) for 90% of those diagnosed, and achieve viral suppression for 90% of those treated by 2020 [18]. Despite significant progress towards controlling the HIV pandemic, at the halfway point, the UNAIDS executive team delivered a ‘wake-up call’ to for those dedicated to global HIV response, acknowledging that both efforts and funding required to support attainment of the 2020 targets were noticeably waning [19].

The PEPFAR Roadmap for Shared Responsibility elevated the spirit of PPPs by emphasizing responsibility: signifying that every sector and every individual organization plays a contributing role to foster vitality of a social ecosystem [20]. PEPFAR defines PPPs as a collaborative endeavor that combines monetary and in-kind resource investments from the public and private sector to address PEPFAR’s HIV/AIDS prevention, care, and treatment goals through active engagement that transcends prescribed contractual obligations. Unlike many procurement-based PPP contracts, the PEPFAR-supported PPPs utilize a Memorandum of Understanding (MOU), executed by public and private authoritative executives, which outlines the collaborative goal and primary partner resource commitments. The Director of Centers for Disease and Control and Prevention (CDC) Africa, Dr. John Nkengasong, described PPPs as a critically important paradigm shift in global infectious disease prevention efforts [21]. Within PEPFAR, there is support from private and public sectors highlighting the valuable role PPPs play when striving to resolve health threats in low-resource settings [22–24].

There are few public health PPPs that focus on investments in the development of personnel who conduct the core lab functions critical to the UNAIDS Scale Up agenda. The strength and capacity of laboratories to provide accurate diagnostic tests for timely case detection, treatment initiation, and maintenance to achieve viral suppression is integral to PEPFAR’s success [25]. However, strategic plans to attain 90-90-90 targets have overwhelmed many public health laboratory systems in sub-Saharan Africa due to increased demand for early infant diagnostic (EID) testing of HIV-exposed infants and viral load (VL) monitoring of patients on antiretroviral therapy [26]. Despite system-level demands, PPPs launched in laboratory settings have become increasingly more attractive in arenas where infrastructure development and healthcare service contracts are losing favor [27].

While PEPFAR 3.0 articulates action agendas in four key areas: impact, efficiency, partnerships, and sustainability (described as operating “in lock step with partner countries as they assume greater responsibility for controlling their own country’s epidemic”) [14]; research dedicated to how multi-sectoral collaborative public health networks function in ways that are impactful, optimize partner experiences, and contribute to long-term system improvements and future expansion is scant [2]. Partnerships are recognized as vital in creating sustainable initiatives that are built for long-term durability yet researchers have called for a more profound exploration of how inputs and implementation processes on the partner-level relate to program impacts deemed significant to individual partners as well as the collaborative itself [10]. There are some studies that have begun examining PPP implementation processes. In a comparative analysis examining factors of PPP initiation, Koppenjan identified patterns associated with positive initiation of PPPs concerning transportation infrastructure enhancements [28], and in another, researchers probed insights into the process of implementation by interviewing core actors involved in three Dutch transport PPPs, asking questions relative to project outcomes [29]. However, these studies pertain specifically to infrastructure maintenance PPPs and a need to examine the implementation processes and impacts of PPPs within public health settings remains.

Research exploring PPPs often focus on comparing partner characteristics, values, and tensions that occur in the midst of quantifying collaborative success. Relatively unchartered territory is the investigation into PPP outcomes reported by diverse collaborators based on their vantage point of the dynamic implementation experience [8]. In this work, we examine recollections of the PPP experience among a diverse network of laboratory-based implementing partners using a Theory of Sustainability lens [30], with the aim of answering the following evaluation questions: 1) how do planned resource and input expectations relate to observed project outcomes? And 2), how do outcomes relate with stakeholder perspectives regarding PPP model sustainability? The evaluation approach is based on principles of Participatory Action Research which centers on reflecting on learned knowledge in order to influence practice [31]. Key stakeholders were interviewed about their collaborative experience as a means to better understand interactions that served a purpose, endured, failed, or perhaps changed. Themes from the responses present rich information for global health leaders to understand a more comprehensive concept of effective PPP implementation practices and effective alliances that can support lasting change [32]. The case network design allows for a collective analysis of themes that emerge across network partners aiming to advance a vast global health intervention agenda within the context of public health laboratory systems strengthening.

PPP case network

In 2007, OGAC, the U.S. CDC, and Becton Dickinson and Company (BD) launched its first PEPFAR supported PPP [33]. Since then, two additional PPPs have been established: one with Roche in 2013 [34], and most recently with Siemens Healthineers in 2014 [35]. The PPPs are linked by a charge to strengthen laboratory systems and enhance workforce capacity to accelerate progress towards attaining the 90-90-90 targets accomplished through diverse training, mentoring, and accreditation efforts. Unified by a common goal, each laboratory-based PPP in the network was distinct. As shown in Table 1, PPPs varied in terms of: the adopted 90-90-90 target, training approaches implemented, modalities employed to engage laboratorians and partners, in addition to project scope, longevity, and level of committed resources.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of three lab-based PPPs by history, scope, and PPP focus.

https://doi.org/10.1371/journal.pone.0254495.t001

Methods

The methodology of this study was guided by the systematic CDC framework for program evaluation, and foundational guidance for evaluating partnerships [41, 42]. Admittedly, the methodology may present trade-offs in terms of generalizability, yet it serves to optimize the relevance, meaning, and overall utility of findings. Investigative team members from CDC played an integral role in operationalizing the evaluation protocol, selecting the instrumentation, and providing the eligible stakeholder sampling frame. The roles of the external evaluation team members were to secure ethical clearance for the interviews, perform subject recruitment, data collection, coding, analysis, and reporting activities independent of the lead CDC research collaborators. Together, the larger investigative team reviewed the coding themes for more nuanced interpretation of study results and implications for future PPP plan guidance.

Sample

The eligible stakeholder sampling pool was identified by CDC and included stakeholders involved in PPP program operations, which includes management, program staff, partners, funding agencies and coalition members [46]. This group bears PPP program implementation experience that has been recognized as an important, yet limited source of PPP evaluation data [1, 47]. Professionals identified by the CDC International Laboratory Branch as key stakeholders of the lab-based PPP network carried out global, administrative, or program level design or implementation roles within the three PEPFAR-laboratory strengthening PPPs. Table 2 provides a descriptive summary of stakeholders included in the sampling frame.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Key stakeholder practice settings and groups recruited for interviews.

https://doi.org/10.1371/journal.pone.0254495.t002

Semi-structured interviews were conducted in-person and via telephone from April through June 2018. Respondents did not receive incentives for participation. Interviewers obtained verbal consent before beginning an interview. To maintain confidentiality of respondent information, investigators provided assurance that results would be reported in aggregate form and not stratified by partner organization, setting, or operational level.

Analysis

Interview transcripts were reviewed to identify and characterize emergent themes corresponding to PPP inputs and outcomes related to sustainability using applied thematic analysis [48]. Key concepts were identified as initial codes. Coding definitions were deductively operationalized through an open coding process to ensure that each code was distinct and then systematically applied to transcripts. Each transcript was independently coded by two evaluation team members and themes were established by combining codes and identifying emerging patterns. Inter-rater reliability was measured by averaging Cohen’s kappa across codes as well as the average percent agreement. Inter-rater reliability was calculated using ReCal2 software [49]. The following primary techniques were incorporated into the analyses to enhance trustworthiness of the findings: (a) minimum of two investigators involved in analysis; (b) triangulation of themes within and across cases [50]; (c) partner engagement in evaluation design; (d) in-depth thematic descriptions; and (e) clarification of preliminary analysis report with CDC partners [51].

Results

In total, 23 in-depth, semi-structured interviews with key stakeholders were conducted by the evaluation research team. All three lab-based PPPs were represented by stakeholder participants in the study sample. The average Cohen’s kappa to measure concordance was 0.90 with an average of 96% agreement for all codes. Given the goal of this evaluation research study was to explore broad implementation experience related to PPP impacts and issues of sustainability among a diverse network of PPP partners, results are not stratified by partner setting or type. The stakeholder interview guide covered items probing PPP implementation experience and perceptions of impact on laboratorian trainees and system capacity to attain milestones of the HIV/AIDS intervention scaling agenda and potential for sustainability and replication.

Three dominant thematic categories were identified by the evaluation research team regarding stakeholder perceptions of PPPs as a mechanism to advance PEPFAR’s agenda of lab system strengthening and workforce development: 1) perceived impacts of PPPs, 2) keys to success, and 3) challenges and proposed solutions. Results of the thematic analysis and emergent themes and sub-themes are presented in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Thematic analysis presented by coding categories.

https://doi.org/10.1371/journal.pone.0254495.t003

Discussion

This innovative evaluation addresses a gap in scientific literature by retrospectively exploring diverse stakeholders’ perspectives of PPP implementation experiences relative to inputs, impacts and sustainability potential. The unique contribution of this qualitative investigation captures in-depth perspectives often neglected in evaluative research related to the vantage point of PPP stakeholders with post-implementation experience, especially within the context of strengthening public health laboratory systems and training the workforce in order to catalyze HIV/AIDS intervention scale up goals [1, 4, 24, 52].

PPPs in global health have achieved remarkable successes in critical areas such as vaccine and essential medicine distribution, healthcare infrastructure and maintenance support in which governments and nongovernmental organizations acting alone have struggled [16, 53]. However, PPPs that specifically target the advancement of laboratory staff and networks, an indispensable component of global health programs, are rare. One overarching theme identified in this study was that stakeholders maintain positive beliefs that PPP project inputs contributed to a wide range of laboratory system improvements. Outcomes included greater operational efficiency, improved communication, and enhanced capacity in terms of the workforce, the laboratory network, and of the public health system. This theme aligns with research supporting that previous PPPs have extended intervention and health care treatment reach, in support of the second 90 UNAIDS’ target, resulting in both health system cost-savings and enhanced specimen processing efficiency [24]. In another study, HIV patients gaining access to a private support program sustained by a PEPFAR-PPP resulted in public resource utilization savings and effective viral suppression among enrollees [53]. While evaluation of quantitative impacts of laboratory-supported PPPs was not the aim of this study, the emerging theme suggest stakeholders felt that PPPs strengthened systems and supported progress towards achievement of the UNAIDS 90-90-90 targets. This theme is similar to a survey study of implementing partners recruited from a network of 15 unique community sustainability-focused multi-sector partnerships who were asked to rate both partner-level and collective-group benefits associated with collaborative engagement. The researchers found that respondents with more favorable ratings of their respective organization’s implementation strategy capacity, exemplified by the ability to plan, organize, balance knowledge and resource contributions, was positively linked with greater partner-level and collaborative-wide benefits, including greater progress made toward overall community-wide sustainability goals [54].

In our study, stakeholders identified distinct capabilities among implementing partners attributed to the public or private sector that contributed to system improvements that persisted post PPP-implementation. For example, the ability to access technical expertise in laboratory technologies and achieve greater operational efficiency was noted with respect to private partner impacts; whereas the longstanding reputation of public health leadership and capability of leveraging a widespread global network for influence were noted as valuable contributions facilitated by public partners. These findings align with attributes of respective public and private partner contributions recognized in relation to project outcomes identified in Roehrich’s systematic review of PPPs [4]. Our results are similar to scientific findings that show an equitable balance of inputs and complementary partner contributions is linked with positive collective impact [54]; however, more nuanced investigation focusing on individual partner-level contributions and impacts, as well as assessing specific implementation strategies activities would yield more insightful PPP formation and performance guidance to optimize sustainability.

Scientific support to advance PPP implementation science must keep pace with the expanding presence of hybrid global public health PPPs. Our study findings offer important identification of collaborative conditions that can threaten effective PPP performance and sustainable improvements. Emergent themes indicate that stakeholders demonstrate a remarkable capacity to reflect on implementation experiences that initially presented as challenges, yet they offered pathways into actions by which coordination of resources, active communication, or additional means were utilized to ultimately overcome such obstacles and in some cases-they were viewed as positive PPP impacts. One example framed as a challenge was cultural incongruence occurring between external mentors being inserted into local laboratories. Research demonstrates that limited interagency relationship-building can have significant effects on collaborative performance [55]. Problem-solving and relationship-building among PPP partners is understudied and our findings demonstrate that it is an area worthy of more attention.

The second overarching theme that emerged from our analysis identified experience of PPP implementation associated with the success of PPPs. Stakeholders’ discussed capacity-building, MOUs, partner knowledge, engagement of leadership and local stakeholders, and mutually recognized goals, as important factors associated with health systems improvement and enhanced laboratory capacity. These findings echo components included in a multi-sector, inter-organizational collaborative framework developed by Quélin and colleagues [56]. The framework emphasizes that rules, norms, and values that exist within any individual partner organization are as of equal importance especially when considering contractual, institutional, and resource/process factors that are forged among organizations engaging in a complex partnership.

What does valuable partnering look like? This is the question PPP researchers such as Barlow, Roehrich, and Wright have asked. Lessons from hybrid PPPs have been recognized as the way to understanding complex project goals that do not map into straightforward performance metrics [6]. In our study, stakeholders reflected on intentional relationship-building building between lead organizations and implementation partners as a critical element associated with PPP impacts. Stakeholders that were more knowledgeable of collaborative partners’ involvement and motivations had greater insights into progress made towards goal attainment than those with an unclear understanding of partner roles, engagement expectations, and limited interactions. Overall, cultivating high quality relationships related to partners and mentors were beneficial in addressing the challenges PPP collaboration presented, which represents the third major theme of our analysis.

Stakeholders recalled specific challenges during PPP implementation that threatened PPP outcomes and sustainability. Examples included times of financial uncertainty, or changes within organizations including personnel or business values, as well as shifts in global markets. This aligns with a study conducted in Nigeria which investigated impacts of PEPFAR funding policy modifications as associated with significant reductions in viral load testing, routine drug monitoring labs, staff employment, and overall interrupted lab services [57]. Our study reveals that misalignment of program strategies and cultural competence of experts interacting with in-country teams posed problems that diminished PPP outcomes and potential for sustainability as reflected in responses. However, stakeholder respondents offered solutions to resolve such difficulties based on their lived implementation experience, which may have paradoxically reframed the original problem to be subsequently positive. This is a phenomenon, identified by Niesten, of how collaborating partners may arrive at their own sense of value in working together [58]. Stakeholders articulated that maintaining open, regular communication and facilitation of a collaborative planning and revision process would enhance PPP collaboration interactions and bolster potential for lasting, local impacts. This finding is similar to the Dutch study of transportation PPPs where lack of interaction was identified as a primary cause of PPP stagnation [29]. Within the realm of global health, research supports that the social value of collaboration can be enhanced through establishment of functional, reliable, and verifiable criteria to initiate, implement, incentivize, and evaluate PPPs [6, 59].

Subsequent research dedicated to PPP formation, conceptualization, and operational norms that preserve and maintain trust, justice, and regard for humanity is imperative [55]. Our results corroborate that while PPPs have clear goals for forging multi-sector, global collaborative partnerships to solve complex population health problems, implementation practices of PPPs must build plans that account for fluid organizational, environmental, and contextual conditions that otherwise may limit project sustainability [4, 56, 60]. The failure to prepare multi-sectoral teams on best practices and collaborative implementation science evidence is linked to negative program outcomes including low workforce morale and the dissolution of partnerships [57].

Finally, the greatest potential to strengthen global health PPPs is to advance evaluation efforts. Defining a mutually agreed upon mixed methods monitoring and evaluation plan that includes key PPP process measures and impacts is imperative for determining the overall health of PPPs and their constituent partners throughout the collaborative cycle. This affords the ability to recognize partner contributions, clarify values, track progress, and institute changes that bear the potential of program implementation disruption [1]. Efficient monitoring of data would allow key stakeholders to understand and quantify the effectiveness of diverse workforce development approaches utilized by respective PPPs, which was not consistently discernible among network stakeholders. Enhanced commitment to standardizing monitoring and evaluation activities would support routine transactions and elevate PPP accountability and transparency practices [27], which were expressed as important to sustained project success. While standardized evaluation approaches to quantify PPP impacts are not widely available; models such as the World Health Organization framework [61] and the P4 Toolkit [44], present pertinent domains to assess complex PPPs within the context of public health. However, these resources require adaptations so they include capture of specific laboratory system strengthening project plans from formation through implementation, as well as strategic plans to address ownership and transitions occurring beyond the PPP life cycle [62–66].

The results of our study highlight the creativity supported by PPPs that join private corporations with public health and government organizations to strengthen overall laboratory capacity and healthcare systems to accelerate the UNAIDS 90-90-90 targets in their countries [67]. However, enhancing laboratory capacity as a system has implications that go beyond disease control [68]. There may be contractual challenges, institutional barriers, and partnering regulations that influence a PPP’s ability to function and operate beyond the original MOU [16, 56]. Recognizing that in-country partners may lack the bandwidth to assume expanded responsibilities that monitoring, evaluation, and reporting activities may introduce if PPP partners are not thoughtful in planning an equitable, supportive plan that is responsive to issues of feasibility and partner capacity [1, 16, 66].

Thus, there is a clear need to understand internal and external structures that maintain the vital indictors of PPPs health, which include respect, justice, and trust among partners so that diverse partner interests, engagement incentives, and expectations are harmonized and conflicts/tension are readily reconciled. Research regarding global health PPP implementation practice improvements would also benefit from advanced assessment into ‘the bridge position’ which essentially refers to role of lead partner in a service-based PPP, and oversight of relationship management and coordinating of interactions among network partners. The extent to which a private partner assumes this role may threaten the vitality of a PPP due to power imbalances, greater potential for improprieties, or a shift in operational values that can upend equitable, balanced collaborative engagement [69]. While these occurrences may not have been disclosed by respondents, framing future studies to capture leadership oversight should be a priority. Subsequent PPP evaluation frameworks must examine performance metrics with more direct role delineation, qualification of implementation activities, and quantification of collaborative advantages/impacts on an organizational-level in order to maximize sustainability [66, 70, 71]. The utility of using a qualitative research approach provided a rare opportunity to explore network perspectives regarding the collective impacts and sustainability of PPP collaborative experiences which provides beneficial insights into the formation of new global health PPP partnerships that remain agile in the dynamic global health arena.

Supporting information

Acknowledgments

The findings and conclusions in this analysis are those of the author(s) and do not necessarily represent the official position of the funding agencies.