News | Nov. 20, 2017

5. Assessment and Program Design

By Jeanne Giraldo Effective, Legitimate, Secure: Insights for Defense Institution Building

Download PDF

With the increased Department of Defense (DOD) focus on building partner capacity (BPC) activities since 2001—ranging from routine, steady-state efforts to encourage democratic and competent partners, to the more extreme cases of Afghanistan and Iraq—most stakeholders involved in security cooperation are conversant with the basic lessons captured in the chapters in this volume. Each country is unique; solutions must be appropriate for partner-nation realities and should not simply mirror U.S. organizations and processes. The influence of external actors, including the United States, is limited; change will be driven from within. There will be actors who actively oppose change. Even when change is desired, it may require cultural adjustments that take time. External actors should have patience; trust and relationship building are essential first steps in supporting change processes. And while the military is important, it is just one tool that a state has at its disposal to achieve national security objectives and serve its citizenry; effective engagement on security issues requires a comprehensive whole-of-government approach.

These lessons seem to overlap with many of those drawn from the international development community’s experience with institutional capacity building over the past decades. Tommy Ross highlights some of these principles in chapter two: “Best practice” approaches should be eschewed in favor of finding the “best fit.” Change is fundamentally political; powerful stakeholders may see their interests undermined by change and oppose it, thus external support for a reform effort should be informed by a political economy analysis. Partner ownership is essential for successful reform. And finally, defense sector reform should be firmly nested within security sector reform.

This apparent convergence on lessons, however, hides some very real differences. By most accounts, the defense community in the United States (and internationally) lags the international development community on the capacity-building learning curve.1 This is not suprising: the sustained capacity-building efforts of the international development community predate those of the defense sector by decades. Notwithstanding isolated U.S. advisory efforts during the Cold War in places like El Salvador and Vietnam, widespread military-military engagement with the objective of building partner capacity is largely a post-Cold War phenomenon. The development community’s critique of its own traditional, technocratic approaches—where outside experts come in with the “right” institutional blueprints and insist on partners having the “political will” to implement them—began in the 1950s and picked up steam in the 1990s, but consensus on an alternative approach only really began to emerge in the late 2000s.2 In contrast, the majority of the security asssistance and security cooperation communities, both in the Department of State and Department of Defense, are still only in the initial stages of developing a critique.

The alternative approach that has emerged in the international development community focuses largely on process issues related to assessment, program design, and implementation, rather than the content of reform. It has been developed by practitioners in a wide range of institutional reform areas, including public sector management reform, security and justice sector reform, and rule of law reform. The DOD’s Defense Institution Reform Initiative (DIRI) has also piloted and validated similar practices for defense institution building (DIB) since 2010. Arguably, the somewhat independent convergence of so many different communities on the same good practices reinforces their validity.3

This chapter summarizes these “state of the art” assessment and program design practices for institutional capacity building, contrasts them to traditional approaches, and describes their applicability to defense institution building. The following section describes how the apparent convergence on the lessons identified above can mask important differences in how these lessons are conceptualized and put into practice, particularly at the assessment and program design stages. As a result, policymakers, planners, and practitioners may think they are in agreement on the basic principles of DIB (and security cooperation), but find they have radically different ideas about how to plan for and implement these activities.

The next section describes the innovations made in DIB programming since 2010, based on an understanding of the failures of traditional, technocratic approaches to institutional reform and lessons learned from previous efforts to execute DIB within the traditional security cooperation paradigm. The challenge was not only to devise a new results-oriented approach appropriate to the DIB problem set but also to carve out space for the exercise of the new approach. This section highlights some of the obstacles to effectiveness posed by traditional security cooperation practices and the minor accommodations to these practices that have been made to enable effective DIB programming.

The chapter concludes by discussing the challenges that lay ahead as DIB moves from being a specialized program offered to a subset of interested and priority countries to a more mainstream part of security cooperation. Care will need to be taken to ensure that the gains made in understanding and operationalizing the requirements for effective DIB—the assessments, program design, and implementation “good practices” identified in this chapter—are not lost. At the same time, there are opportunities for security cooperation writ large to become more effective and sustainable with the mainstreaming of DIB, but only if this is understood to include more than just the expansion of DIB programming. It will require a nurturing of the initial tentative steps being taken to make DIB considerations a more integral part of the policymaking, strategic planning, and security cooperation planning processes.

International Experience with Institutional Reform: New Approaches to Assessment and Program Design

The International Security Sector Advisory Team (ISSAT) aptly notes that security sector reform has three key dimensions—technical, political, and holistic—and should be driven by a general guiding principle of “local ownership.” 4 This section describes how apparent agreement on basic principles in these areas (e.g., “best fit, not best practices” to describe how technical knowledge should be applied) can hide very real differences in how each of those insights is conceptualized and put into practice. In general, the difference is between a more narrow interpretation of the lesson that leads to minor adjustments in how institutional reform is undertaken, on the one hand, and on the other, a more fundamental critique of traditional approaches, with significant implications for how assessment and program design are conducted. The international community’s long path to developing alternative approaches arguably stems in part from an evolutionary process where, initially, only tweaks were made to the traditional approach; when these proved ineffective, more radical changes were attempted with better results—hence the current consensus on the approaches described below.

The Technical Dimension of Reform: Best Fit, Not Best Practices

In the United States, discussion of the role of technical expertise in the DIB workforce has been deeply colored by the experiences in Iraq and Afghanistan, where military commands were in charge of institutional capacity-building efforts. In many cases, non-experts in defense management (active duty military or retired military hired as contractors) resorted to “mirror imaging” of U.S. models, which proved inappropriate in the partner-nation context. Some reacted by drawing the assumption that “real” experts—individuals who had performed defense management functions within the U.S. system—would be better-placed, after some advisory training, to avoid these pitfalls.5 Others noted that, given the size, scale, and level of sophistication of the U.S. defense bureaucracy, U.S. experts would likely have an in-depth knowledge of individual parts of their own system, but not the overall view of systems and processes needed to provide advice to partners pursuing reform. Instead, experts from countries with smaller defense sectors and U.S. experts who have studied the diversity of international experiences with defense management, would be better placed to act as DIB experts.

U.S. policymakers expressed their faith in technical expertise by calling for the use of baselines of defense management best practices as a guide for action. In particular, detailed lines of activities—i.e. frameworks defining the technical details of functions within each defense pillar, such as resource management or human resource management—were called for as “menus” from which security cooperation planners could choose. This inclination toward DIB templates came at a time when the international community had largely rejected templates, after decades of relying on international best practice frameworks that proved largely ineffective. Just as the U.S. experience in Iraq and Afghanistan had resulted in the mirror imaging of U.S. processes, the development community’s experience had resulted in the mirror imaging of international best practices (institutional isomorphism). Formal laws, documents, and organizations were created that looked good on paper, but had little impact on how things functioned or, if they had an impact, did not produce the desired outcomes.

In some cases, this disconnect between form and function could be attributed to partners adopting best practices under pressure from the international community, without the partner intending to implement said practices. In many cases, however, partners willingly adopted best practices in the hope of achieving improved governance and were disappointed with the results. This can be attributed in part to a major flaw in the best practices approach: templates typically depict an ideal system of all the best practices thought to possibly make a difference, but which no country (not even the most advanced) displays. This has led experts to acknowledge that the goal should be “good enough governance”—identifying the minimum subset of practices necessary to produce the desired outcomes in any particular context.6

In addition, experts acknowledged that changes in formal processes might not lead to the desired results because informal factors (e.g., status, personal relationships, beliefs, individual incentives) mattered as much or more for achieving the desired outcome. The importance of informal practices and other country-specific factors in shaping the impact of institutional changes meant that experts had to show a greater humility in their approach and work much more closely with partner-nation personnel to identify whether and how different good practices might be tailored to partner-nation realities to produce the desired outcomes.

Among other things, this meant eschewing the traditional best practices gap analysis approach to assessment and program design. This approach compares existing partner-nation practices to best practices, identifies an (often long) list of gaps between the two, and recommends working through the checklist of identified gaps until a modernized system is in place. However, while checklists and templates can prescribe specific and sequenced steps for addressing complicated technical problems (e.g., rebuilding a carburetor or preparing an airfield), they are less appropriate for complex problems like institutional reforms (sometimes called “wicked problems”), in which human beings are involved and the context significantly affects results.

When context matters, and the goal is to find the best fit rather than import best practices, assessments need to center on defining the relevant aspects of the context rather than gaps in best practices.7 Two complementary approaches have emerged in this regard: a problem driven approach which focuses attention on the specific desired outcomes of reform in a particular context (the “problem” to be solved) and a baseline description of current practices (the “as is”) that should fundamentally shape thinking about the kinds of reforms that might be feasible and necessary.

The problem driven approach provides the “North Star” to orient reform efforts. Instead of identifying the lack of best practices as the problem to be addressed (as in traditional approaches), the focus is on identifying the key problem that the partner cares about (and thus will be motivated to address). It is a facilitated conversation with the partner about shared strategic objectives and priorities, and the extent to which the performance of their defense management system is affecting the realization of these objectives. In some cases, the “pain points” may be very clear from the start, even if it will take stakeholders time to collectively identify their root causes and possible solutions. In other cases, the framing of the problem and possible solutions may require some refining over time in an iterative process that is the hallmark of approaches for working in volatile, uncertain, complex, and adaptive environments. A key focus is identifying the necessary and sufficient ingredients required to solve the problem, which often means addressing not just institutional processes but the broader system within which actors operate.

In addition to identifying the problem to be solved, DIB practitioners work with the partner to develop a baseline (“as is”) assessment of the existing formal and informal “rules of the game”—who does what to whom and why and with what effect—to identify where change might be necessary and feasible. In contrast to a best practices approach, change is not about experts designing new processes and insisting the partner leadership have the political will to implement them, but rather about determining how it might be feasible to get large numbers of individuals to behave differently. Understanding existing approaches—their strengths, the opportunities for change, and the perceptions and interests of the actors involved (e.g., decision makers, analysts, implementers, users)—is key to the problem solving effort.

The Political Dimension of Reform: Thinking About How Change Happens

A conclusion reached by most capacity builders is that change affects the balance of power, and will frequently be resisted by those who benefit from the status quo. While perhaps obvious, this insight represents a departure from a more technical theory of change embodied in early public sector management reform (and implicit in many security cooperation training and education approaches to capacity building): provide partners with an understanding of “what right looks like” and they will make changes in their organizations or their behavior to produce better outcomes. Unfortunately, while education often serves to help an individual understand the dysfunctions of the system, it usually does not empower or incentivize them to make change.

When change did not come automatically with the imparting of knowledge, or was resisted, external actors began to revise their theory of change, concluding that political will, and not just knowledge, is essential for change. In response, external actors operating at the policy level have tried to influence a partner’s political will through conditionality, diplomacy, or a combination of “sticks and carrots.”

At the program level, the reaction to the need for political will has varied. Technical experts sometimes insist that the political aspects of reform are outside their duties or “above their pay grade”—their job is to share knowledge, but it is the partner’s (or donor’s) responsibility to do something with that information. At other times, technical experts take a more proactive approach but one that is based on a fairly technical, linear, and hierarchical understanding of the policy formulation and implementation process: leadership endorsement is secured before beginning a project; the relevant staffers are engaged over a period of time (often years) to teach them best practices and have them apply this knowledge to the creation of a document or new process; leadership then publishes the resulting product, creates a strategic communications plan to ensure it is understood by implementers, and monitors implementation. The timelines for this approach are often divorced from political reality, and are driven by the technical complexity of the processes being taught and the amount of time required for this process.

Unfortunately, these approaches to tackling the political dimension of reform often yield changes in the form of partner institutions, but not their function. Outsiders can force the formal adoption of new laws or procedures through conditionality or through a sense of obligation (for instance, as part of a package of technical support), but neither result in real change. Despite the importance of “champions of change,” especially in hierarchical partner-nation militaries and bureaucracies, leadership endorsement—or the political will of a single individual—is also usually insufficient to affect change. The scope and process of change tends to be much broader and messier than envisioned by the linear, hierarchical approach: if true changes in the ways of doing business are desired, the broader set of stakeholders expected to adopt, implement, and institutionalize new practices needs to be consistently considered throughout the process. Coalitions for reform need to be built, and may come together in different ways and at unexpected speeds in different settings.

To address these shortcomings, the international community has stressed the need for a political economy analysis (PEA) to understand the incentives and power of a wider array of stakeholders who might support or oppose change. In some cases, dysfunctional processes may exist not because they serve elite interests, but because factors that affect human behavior and decision-making—inertia, fear of the unknown, discounting of the future, limited time and attention span—are often at work, and must be overcome through a reform process that deliberately targets these issues. In other cases, change might serve the collective good and make everyone better off but it is not in the interest of individuals to unilaterally take action to solve the problem. Understanding the factors that contribute to the preservation of the status quo and how these might be changed is essential for successful reform.

Though it represents a broader effort to tackle the politics of reform, more than a focus on leaders and political will, PEA by itself has not been a sufficient tool for adequately incorporating the politics of change into program design and implementation. There are at least two plausible reasons for this: first, stovepiped functions in many donor agencies mean that assessments are conducted by one set of actors, and programming is shaped by another set of actors who tend to choose a stock program with little regard for initial assessments. Second, PEA conducted during the assessment stage frequently only offers limited insight into stakeholder interests. Instead, PEA must be thought of as an ongoing process that continues during the design and implementation phases of a project.8

In addition to incorporating PEA as a central feature of their new approach at the assessment, design, and implementation phases, external reformers have responded to the politics of change by emphasizing the central importance of engaging local stakeholders in the process from the start. From this perspective, “problem definition is an inherently political process of negotiation and priority setting.” Problems and their potential solutions are not determined externally by expert assessors, but should be the subject of debate among local stakeholders, supported by experts. In addition, engaging stakeholders from the start “helps uncover the real incentives and interests of the actors involved in conducting . . . reform change and finds a compromise between them.”9 Moreover, the process of change itself may alter an actors’ interests and stance toward reform, requiring that experts have the flexibility to adapt or iteratively build their programs accordingly.

In this approach, PEA can help the reformer understand the initial space for reform and propose a plausible theory of change. But since change is multifaceted and there are likely multiple paths to the desired outcome, the steps of the process cannot be laid out in great detail in advance. The terrain can be mapped; the reformer knows to avoid the cliff to the left and the waterfalls to the right, but other paths might be pursued sequentially or concurrently, with the reformer having the flexibility to change course when dead-ends are hit or to accelerate change when a clearing is found. This understanding of change (and corresponding program design) is often contrasted with the sequenced roadmap of reforms produced by a best practices gap analysis. If the latter is similar to a train moving down a track hitting milestones at determined intervals, more complex institutional change is akin to a sailboat, tacking according to the prevailing winds but guided by a North Star and problem-driven sequencing of reforms.10 In many cases, the lines between the assessment, program design, and implementation phases become less distinct with the iterative development of knowledge of the technical and political feasibility of different possible solutions over time.

The Holistic Dimension of Reform: Setting a Realistic Level of Ambition

The security sector reform literature rightly stresses the need to understand how parts of the security sector interconnect within broader government, legal, and societal contexts. This need for a holistic perspective, however, is often mistakenly interpreted as the need for holistic action—undertaking an overly comprehensive assessment or proposing an overly ambitious reform agenda at the program design phase. In both cases, the results ironically tend to be far less than what would have been produced through more targeted assessments and projects. As ISSAT notes:

Experience has also shown that comprehensive assessments that aim to cover all actors in depth are ambitious, complex, resource intensive and time consuming. Comprehensive assessment efforts often falter with poor results that assess a limited number of actors unevenly. As a result, comprehensive assessments should generally be avoided, and focus given to areas where there is likely to be a greater positive impact.11

Of course, the challenge is to determine what criteria should be used to identify focus areas of greater positive impact. Traditionally, the best practice framework has provided little analytical purchase in this regard, as even less-than-comprehensive assessments usually produce long laundry lists of gaps. This often results in programming that is overly ambitious—comprehensive reforms to address as many of the gaps as possible. Or, if the limits of the partner nation’s time, energy, and resources for reform are acknowledged, a curiously unambitious approach is taken: Since everything cannot be addressed, an “anything helps” mentality prevails, with external agencies almost inevitably able to find a need for their stock programs among the list of gaps. Programming ends up being the same as it would have been in the absence of an assessment. In both the overambitious and under-ambitious cases, programming is supply-driven—external experts offering up best practice templates for reform or favored individual programs.

In reaction, the international community developed the more diagnostic approach to programming described in previous sections, which is focused on the need to identify achievable and relevant reforms based on partner-nation realities and desired outcomes. This problem-driven approach seeks to understand how a desired functional outcome might be reached, presuming that the introduction of some subset of good practices will be necessary but insufficient to reach the desired outcome. The approach emphasizes flexible problem-solving, acknowledging the constraints of the environment while focusing on the actors who will be responsible for defining the problem and designing, implementing, and sustaining new ways of doing business. Based on what is known about actors’ perceptions and incentives, and a problem’s technical challenges, can a clear theory of change be proposed for how the proposed changes will expand the reform space and fix the functional problem? This diagnostic approach, as the World Bank notes, “implies an emphasis on flexible problem-solving,” conducted iteratively, rather than a formal project design and approval approach with solutions defined in detail in advance; “the traditional distinction between ‘design’ and ‘implementation’ . . . gets blurred – and more flexible instruments are needed.”12

Local Ownership

The preceding pages have described the upending of the traditional, technical approach to institutional reform, shifting from a long-standing model—where experts and their best practices were the driving force behind change—to one where partner-nation stakeholders are rightly understood to be the engines of reform (with external experts acting in a support role). Within this context, the concept of local ownership of reform is easily grasped, but nonetheless is subject to varying interpretations that determine whether the intended result is achieved.

In some cases, the principle of local ownership may be acknowledged, but the political priorities of the donor or the lack of flexibility within donor planning processes may lead to decisions being made and then imposed on the partner. This is particularly the case when external actors are required to produce and adhere to detailed plans early in a relationship, when the problems and solutions have not yet been discussed with the partner.

In many cases, ownership is often equated with the existence of formal agreements between countries and formal governance structures overseeing the work being supported by external experts. However, there is nothing automatic about this equation: it is not uncommon for formal agreements to mask a lack of real partner ownership. In part, this is because formal agreements are often sought at the very beginning of a process of reform and this timing can pose serious challenges for meaningful local ownership. Early on, it is often not clear to policymakers what they are really signing up for, creating the illusion of ownership where there may be very little. This is particularly the case when leaders endorse an initiative, with the assessment and program design phases to follow.

In addition, when a more detailed plan is formalized early (after a truncated joint assessment process), the danger lies in narrowing the potential scope of reform, since dialogue about required changes often leads to more expansive support for reform months into the project rather than at its onset. This can be accommodated if early agreements do not attempt to lock in overly specific milestones, but instead agree to the steps that help actors to deepen their understanding of required changes (for example, conducting “process tracing” as part of the implementation plan). As one author notes:

An iterative process would . . . view an assessment not as a prerequisite to beginning an SSR process but as an integral part of that process. An iterative process would not aim to tick the box of local ownership, but rather see it as a continuum of open discussion between the national government and international partners, regarding expectations, needs, and what is feasible given certain constraints.13

The conceptualization of local ownership as a “continuum of open discussion” also provides a corrective to some notions of local ownership that put the partner completely in the driver’s seat. This may be the case when a stakeholder interprets ownership as asking the partner what they want, as opposed to engaging in an informed dialogue about issues of mutual importance and debating the best strategies to advance these shared objectives. This also takes place when teams of contractors are assigned as advisors and see their job primarily as providing technical advice and keeping the partner happy. In contrast, the expert should add value by providing a transformative perspective, helping partners discover new ways of doing business and supporting reformers in the face of resistance to change. This ranges from introducing challenging new good practices, like infusing transparency into a system as part of an effort to improve effectiveness, to opening the possibility for new approaches by bringing disparate stakeholders together that would never have spoken to one another if left to their own devices.

Defense Institution Building Innovations

As DOD looked to take DIB global under the DIRI program in 2010, it modified the traditional DIB paradigm to reflect the critiques and alternative approaches outlined in the first part of this chapter. Notably, these changes were not motivated by a desire to apply lessons learned from the international experience to DIB; in fact these international lessons learned were being articulated over the same time period as the DIB innovations (2009 and following), and the similarities between the two processes only became evident much later. Instead, the DIB innovations were driven by an analysis of the practices required for strategically relevant outcomes and built on lessons that had emerged from the experience of the Warsaw Initiative Fund (WIF)-DIB program which was already in place (but limited to supporting members of NATO’s Partnership for Peace program).14

The WIF-DIB “playbook” in place in 2009 was characterized by a fairly traditional, technocratic approach to capacity building. A DIB team would conduct a technical “requirements determination,” assessing (usually over a week) the partner’s defense organizational structure and its processes in strategy, policy, and planning, resource management, human resource management, and logistics, as compared to international best practices. This typically generated a long list of gaps that formed the basis for a sequenced roadmap of activities—a menu that security cooperation planners used to match providers to activities, working their way through the list based on U.S. objectives, partner interest, and availability of funding.

The DOD’s creation of the DIRI Program in 2009 significantly advanced both understanding and the practice of defense institution building. This section describes three of the fundamental changes that were made as DIRI hit its stride. First, DIRI expanded the scope of DIB to capture all areas of the defense institution that needed to be targeted to see real improvements in governance and management. Second, DIRI began every DIB project with a scoping phase, where all relevant stakeholders would determine what was achievable and relevant in a given partner-nation context—moving from a technical, best practices approach to a politically-informed approach. Third, DIRI established an approach to program management, project design, and implementation that allowed the long-term perspective and short- to medium-term flexibility required for effective DIB.

Each of these innovations had implications for how DIB programs related to other security cooperation activities and to the traditional security cooperation planning process. Understanding this logic will be essential as DOD seeks to institutionalize DIB as a standard part of security cooperation programming. Ideally, the process will meet the requirements of all stakeholders, while retaining the planning and programming required to make DIB effective.

Defining DIB

DIB was initially characterized as a ministerial capacity-building program contributing to broader objectives of defense sector effectiveness, affordability, transparency, accountability, and responsible civilian control.15 Building on WIF-DIB practice, four main pillars were identified as key to effective governance: the management of ideas (strategy, policy, and planning), money (resource management), people (human resource management, including defense and military education), and things (logistics). Civil-military relations and interagency coordination, which were treated as separate focus areas under WIF-DIB, were incorporated as cross-cutting principles potentially applying to all DIB activities.

In some quarters, “ministerial capacity building” was viewed from a traditional, technocratic perspective, focusing on sharing knowledge with Ministry of Defense (MOD) officials to improve their ability to manage the defense sector (e.g., providing policy guidance or managing money). However, viewed from a broader political and holistic perspective, it was clear that this narrow approach would not increase the MOD’s capacity to effectively oversee ideas, money, people, and things, and thus could not contribute meaningfully to solving the problems partner nations faced. To do that would require understanding—and potentially influencing—the broader governance and management system and set of power relations within which the ministry operated.

In any defense sector, the MOD is only one part of the overall defense management system—a key fact that was acknowledged by broadening the scope of DIB to include “the ministerial, general, joint staff, military service headquarters, and related defense agency level, and when appropriate, . . . other supporting defense entities.”16 Focusing only on the MOD’s ability to do their job would make sense if defense management processes were structured effectively and only lacked competent ministry officials. However, in most cases, effective DIB requires a review of all the steps in the “results chain” (looking across the “levels” of the defense sector) to determine where breakdowns are preventing effective processes and to identify required action. The organizational capacity of the ministry is important, but should not be confused with the overall institutional capacity of the defense sector, which refers to “the people, organizations, rules, norms, values, and behaviors that enable oversight, governance, management, and functioning of the defense enterprise.”17

This caveat about the role of the MOD is true everywhere, but is particularly important in partner nations where considerable power resides in the military services, rather than in the MOD or even the general command or joint staff. Introducing new ways of operating in these settings often requires changing power relationships and empowering a ministry that may be reluctant to assert its authority to lead the defense sector toward more effective and accountable practices. Understanding these power relationships, possible sources of leverage, and urgent problems that might serve as the opening for working collectively toward new ways of doing business is a necessary first step.

In this system of systems approach to defense management, the ministry is the preferred entry point for engagement, even if those interactions eventually lead to primarily working with other interlocutors as the main change agents. (In some cases, depending on the problem to be solved and the power of the ministry, engagement with the MOD may not be necessary.) However, engaging at least initially through the ministry represents not only a normative stand on the importance of civilian control, but also the acknowledgment that the ministry is frequently potentially responsible for the changes needed to create new ways of doing business. The MOD is, after all, at least formally responsible for representing the defense sector to the rest of the executive branch, the legislature, and the international community, and interacts with those actors on fundamental legal, policy, and resourcing issues. This includes working with the Ministry of Finance, Ministry of National Planning, and legislative bodies on resource management issues, and with other ministries in the security sector to coordinate and deconflict roles, missions, and service delivery.

In addition to this outward-looking role, the MOD is responsible for ensuring that guidance from executive and legislative actors shapes the actions of the defense sector on key issues related to the design of defense sector processes. From its key position above the military services, and as representative of the national interest, it can advocate for solutions to collective action problems that military services—motivated by their individual interests and perspectives—may not be able to reach. Designing effective solutions for the defense sector requires the MOD to work with key stakeholders in the general staff, service headquarters, and elsewhere to ensure the right design and implementation.

This broad understanding of institutional capacity development, requiring a partner to look across the levels of the defense sector to understand what might need to be changed to improve performance and solve specific challenges, demands a similarly integrated approach from DIB providers. However, this runs contrary to the counterpart approach to engagement that often characterizes security cooperation (e.g., the Defense Logistics Agency in the United States engages with the corresponding agency in a partner nation). This approach not only runs the risk of mirror imaging, but also is likely to contribute to the development of only one part of the system—often with little effect if reforms in other parts of the system are required.

Assessment: “Scoping” is the Key

While the traditional paradigm identified a best practices gap analysis as the first step in a DIB project, this was replaced with a scoping phase that engaged key U.S. government and partner-nation stakeholders to identify shared objectives and priorities, areas where institutional performance hindered the realization of these objectives, and feasible and realistic entry points for DIB projects that could advance those objectives. The term “scoping” was selected to emphasize that the necessary start point was not a comprehensive assessment of the partner’s defense institutions—an undertaking most partners were not willing to engage in—but rather a process that narrowed or scoped the project to target the most strategically relevant and achievable reforms.18

Reforms were considered to be “achievable” only if they advanced partner-nation objectives and priorities. Understanding this is the point of departure for the scoping process. This emphasis was subsequently validated by a large-scale RAND study, What Works Best When Building Partner Capacity and Under What Circumstances, that identified support for partner priorities as the single most important factor affecting the success of BPC efforts.19 This is a central focus of desk study activities where U.S. government stakeholders are asked to share not only U.S. strategic objectives and priorities, and U.S. perceptions of institutional shortcomings that were affecting the realization of these objectives (performance gaps), but also their understanding of how different partner stakeholders viewed their strategic priorities and institutional issues. Understanding the fundamentals of how relationships were structured—among MOD, General Staff, Service HQ, and commanders; between the defense sector and the security sector; and between the defense sector and the rest of the government, legislature, and society—is also a big-picture issue addressed during the scoping phase.

A successful DIB project usually requires engaging with the partner at the highest level possible in order to understand its objectives and priorities, to link any institutional reform efforts to those objectives, and to lay the groundwork for the high-level support that would be needed to affect institutional change. Since defining a program should be a political and strategic process, similarly high-level stakeholders need to be consulted on the U.S. government side. This includes the regional office of the Under Secretary of Defense for Policy (OSD-P), which sees DIB as a key engagement tool to build relationships with counterpart MODs, as well as State Department and U.S. Agency for International Development officials on the ground (e.g., ambassador, deputy chief of mission, and pol-econ and pol-mil officers). Stakeholders at Combatant Command (CCMD) headquarters, the Office of Security Cooperation Chief, and the Senior Defense Official-Defense Attaché are also consulted, and may have different, although usually complementary, perspectives on U.S. and partner objectives.

Typically, a conference call between DIB scoping experts and all key stakeholders—OSD-P, CCMD, and embassy personnel—is held as soon as a DIB request is received. This allows any differences of opinion to be resolved to avoid competing mandates from different stakeholders. It also maximizes the information available for understanding the partner, as different stakeholders are likely to have insights on different aspects of the partner-nation reality. In addition, international actors working in the country are consulted, not just to deconflict activities, but also to benefit from their insights on working with the partner.

As in other policy areas, U.S. objectives for defense sector reform in a given country are usually stated in a generic way and are often the same for all countries (e.g., the desire for an effective, affordable, accountable, and professional defense force). Since this is the case, the challenge is to derive more specific objectives by understanding overall strategic aims and priorities and, through a facilitated dialogue on the topics described above, work with stakeholders to identify the kind of DIB project that would best advance shared partner-nation and U.S. objectives. Often this is an iterative process, with rough general answers identified early on but greater fidelity added as the DIB program begins work with partner-nation stakeholders (and continues discussion with U.S. stakeholders) on topics of interest.

Comparison to Traditional Security Cooperation Planning Approaches

The traditional approach to security cooperation is often a highly stove-piped, menu-based one: the policymaker or planner assesses the requirement, reviews the tools in the security cooperation toolkit, and decides which are needed. Given the standardized nature of most security cooperation programs, the policymaker or planner can look at the available menu of courses or training opportunities, and select the most relevant one based on his or her understanding of what is required. Not infrequently, the menu is offered directly to the partner, who identifies items of interest. The provider then offers the standard course, unable to tailor it in advance to partner-nation requirements because the specific requirement is unknown or uncommunicated to the provider. Good providers will make adjustments on the ground to try to improve the value of their offering, but these are often highly constrained by the circumstances.

DIB programs, in contrast, are highly tailored to partner-nation realities and strategic objectives. Conducting this (initial and ongoing) tailoring requires a facilitated dialogue between policymakers, planners, and DIB experts, as well as partner-nation stakeholders. While general topics and focus areas can be provided to policymakers to give a sense of the scope of DIB work, a menu-driven approach typically fails when applied to DIB.

Program Management, Project Design, and Implementation: Developing a Flexible, Iterative Approach

When the Office of the Secretary of Defense (OSD) decided to expand DIB in 2009, it made a deliberate decision not to delegate the programming of DIB activities to CCMD planners, opting instead for a more centralized program management structure that would work closely with OSD. This was motivated by several lessons learned from the WIF-DIB experience: that the strategic-level entry point of DIB programs would require strategic decisions by OSD regional offices and high-level interaction with the partner, thus making it advisable that the DIB program have privileged access to those offices; that the tactical and operational focus of most CCMD planners would not lead to effective DIB planning; and that existing security cooperation providers and tools available to the CCMDs for planning DIB were largely oriented toward an educational mindset, rather than persistent engagement over time, to achieve collectively defined improvements in defense management. Finally, even if individual providers could make the shift to a more project-oriented approach, the WIF-DIB experience demonstrated that progress in DIB required an integrated approach at the country level, designed and overseen by DIB experts.

The DIRI program was tasked with devising an effective approach to institutional reform. The program would provide persistent engagement with a partner, engaging as frequently as the partner desired for up to two weeks per engagement. The program did not provide education or training, but took a problem-driven approach, working with the partner to determine the scope of the effort and project objectives (which part of the defense management system would be targeted for improvement, how, and why). This would require developing a methodology for the most effective engagement in institutional reform—the approaches to assessment and program design described in the first half of this chapter—and a management structure to operationalize the approach, learn lessons, and adapt. This included an emphasis on creating a strong linkage between the otherwise stove-piped steps in a planning process—with DIB management: contributing to strategic discussions about realistic and relevant DIB objectives at the policymaker and planning levels; designing and overseeing projects accordingly; and then ensuring feedback loops that allowed for adjustments along the way. Both the international institutional capacity-building literature and the experience of other security cooperation providers stress the importance of clear linkages between the assessment, design, implementation, and monitoring and feedback phases, and lament their loss when multiple agencies and bureaucratic processes intervene.20

Responding to palpable “assessment fatigue” at the partner-nation and U.S. Embassy level—long, intrusive assessments that answered the bureaucratic imperatives of donor agencies but did not produce any actionable support for the partner—the program developed an approach that allowed action to begin almost immediately on priority issues. The dates, topics, and stakeholders for a follow-on engagement would usually be settled with the embassy and partner before departing at the end of an initial scoping visit. To allow for U.S. policy feedback, partner-nation ownership, and the broadening of the reform space that would take place over time, a project might begin with two complementary tracks: action on quick-win or priority items identified by the partner (and of interest to the United States), and activities designed to contribute to the collective definition of problems and possible solutions. A deeper dive into technical details of systems and understanding of stakeholder interests (e.g., process tracing) that was previously part of an external assessment, are instead conducted by DIRI as part of the project, allowing relevant partner-nation stakeholders to jointly define problems and their solutions. Multi-stakeholder workshops are particularly effective at launching this process, bringing together multiple actors who normally communicate infrequently, and whose perspectives often depend on where they sit in the defense bureaucracy, to discuss key challenges and next steps. Ideas of spiral development, quick wins, and pilot projects are incorporated to generate momentum for reform or, alternately, to fail quickly and move on to more promising approaches, while having learned more about the system by taking the initial action.

Understanding that institutional change is most likely to succeed when it responds to politically urgent priorities, DIRI was structured to be able to respond to emergent requirements from partners, rather than requiring they wait until the next planning cycle. For instance, a new head of state who is elected on a security platform would want to initiate change in the first 100 days in office, and the United States would want to be able to rapidly provide support in order to take advantage of the initial window of opportunity for reform (as well as to have the entire length of the administration to consolidate gains).

Program funds are also managed to address some of the dilemmas posed by traditional security cooperation approaches. In many cases, roadmaps for reform are laid out without resources to match. DIRI projects, in contrast, are scoped to ensure that results can be achieved within the level of resources that the United States and partner nation are willing to devote to the project. Flexibility in funding, a key good practice for effective institutional reform, is also a central tenet, in contrast to many security cooperation projects with funding levels locked in for a country—leading to funds being spent, regardless of progress being made. Flexible funding allows funds to be shifted between lines of effort within a country project and between country projects depending on progress being made. In addition, even though funds are “one-year money,” program management practices allow for longer-term commitments to partners, as long as progress is being made. While challenging from a management perspective, this approach allows for an effective, long-term perspective to be taken at the individual country level.

The Way Ahead: Mainstreaming DIB

As of this writing, a number of processes are underway that represent the mainstreaming of DIB. A planned DOD Instruction defines the roles and responsibilities of all security cooperation stakeholders in guiding, planning for, and implementing DIB activities. Congress has been paying increasing attention to DIB since the summer of 2015 when it suspended the delivery of large equipment packages for a number of African countries under the Counterterrorism Partnership Fund on the grounds that the partners would not be able to absorb and sustain the equipment. DIB activities were required to be added to the equipping packages, forcing at least the beginning of an internal DOD debate on sustainment. In addition, proposed changes to the FY17 National Defense Authorization Act in 2016 require DIB activities to accompany every train and equip package—though the intent in this case is not the sustainment of U.S. security cooperation investments but rather the strengthening of responsible civilian control.

While potentially valuable because of the increased attention these efforts direct to DIB, they run the risk of making DIB merely a routine part of a security cooperation process that most stakeholders acknowledge requires some adjustments. Conducting seminars on DIB to “check the box” for a DIB requirement is unlikely to contribute to policy objectives. And even persistent engagement on topics selected from a DIB menu (the old WIF-DIB approach) are unlikely to have an effect unless planned and executed as part of a more deliberate and integrated DIB strategy. Without a coherent, results-oriented approach to DIB activities, these will yield few benefits.

Even with such an approach, many of the objectives associated with DIB—such as the sustainment of other U.S. security cooperation investments—cannot be accomplished through action at the program level alone. Instead, a more profound integration of DIB principles into security cooperation plans is required.

In this regard, the combined actions of Congress and DOD policymakers have helped to create an awareness among security cooperation planners of the need for training and equipping plans to incorporate DIB considerations if equipment provided is to become an operational capability that can be effectively and legitimately employed and sustained. In particular, the idea of a “capability planning package” has resonated with military planners, as it highlights the concept that buying a piece of equipment does not automatically create a military capability; rather, operational enablers and institutional capacity are required for equipment to become an employable and sustainable capability.21

Despite the progress this represents, more work will need to be done to convert this concept into an effective approach. The initial impulse from security cooperation planners has been to attempt to implement the concept in a way that only tweaks the existing equipment-centric security assistance system. Just as past innovations designed to increase the sustainability of U.S.-provided equipment began with a specific piece of equipment and added on a two-year package of spare parts, maintenance, and training to form a “total package” approach, the “full-spectrum capability planning package” is often interpreted as “wrapping DIB elements around a piece of equipment.” Such an approach, however, will only add to security cooperation spending, likely with little effect.

Training and equipping plans made in the traditional way—with little regard for the partner’s institutional capacity to employ and sustain the assistance—cannot suddenly be made sustainable with the addition of DIB programming. In contrast, an integrated approach to DIB and security cooperation would require adhering to guidance provided in Presidential Policy Directive 23 on Security Sector Assistance : “Formal assessments of current partners’ institutional capacity, political will, operational capabilities, and anticipated security sector needs will be conducted as needed as the first phase of strategic planning at both the country and regional levels.”22 This assessment of institutional capacity should not be seen primarily as providing a roadmap for the programming of DIB activities, but as a fundamental input to a strategic approach that includes asking what kind of training and equipping can be effectively employed and sustained given the partner’s institutional capacity (or potential institutional capacity with expected DIB accomplishments during the planning timeline).

Policymakers may decide to provide equipment that is not sustainable for a number of reasons—in order to maintain the relationship or stay competitive in a context where other countries are willing to provide “shiny objects”; to support the U.S. defense industry; or to create a long-term relationship that comes with the sale of major pieces of equipment, whereby the partner becomes reliant on the provider for parts, training, and maintenance. But these competing policy objectives should be acknowledged, and explicit policy decisions should weigh those objectives against the sustainability objective. If still deemed beneficial, more realistic objectives can be set for DIB programs in those countries. While some of these conversations have taken place in the halls of the Pentagon in response to new Congressional requirements, they are by no means guaranteed within the current approach to security cooperation, which sets high-level policy objectives that are not fleshed out in any kind of detail required for effective DIB action. Instead, if DIB programming is to live up to its potential, stakeholders must seek out the opportunities that can be created within current security cooperation planning processes for policymakers, planners, embassy personnel, and DIB experts to engage in this kind of strategic, results-oriented conversation about DIB objectives, program design, risks, and potential payoffs.

Notes

1 A recent study notes that “the main programming and knowledge gap at this time is the lack of information on military sector reform effectiveness.” The International Security Sector Advisory Team, “What Works in International Security and Justice Programming? Preliminary Findings,” June 2, 2015, available at <http://issat.dcaf.ch/content/download/92383/1618536/file/International%20Good%20Practice%20Approaches %20Report%20Release%202015.pdf>.

2 Representative critiques include: World Bank, Public Sector and Governance Board, “The World Bank’s Approach to Public Sector Management 2011-2020: Better Results from Public Sector Institutions,” February 3, 2012; Vivienne O’Connor, “A Guide to Change and Change Management for Rule of Law Practitioners,” January 2015; Organisation for Economic Co-operation and Development, “Improving Security and Justice Programming in Fragile Situations: Better Political Engagement, More Change Management,” April 12, 2016; Lisa Denney and Pilar Domingo, “Context-relevant, flexible and transnational programming. The future of security and justice?” December 2015.

3 Similar good practices have been advanced over the last half decade by others within the U.S. defense community, suggesting an even broader applicability in areas such as counterinsurgency campaign planning. For example, see Jack D. Kem, Design: Tools of the Trade (Fort Leavenworth, KS: U.S. Army Combined General Staff College, 2009).

4 International Security Service Advisory Team, “SSR Overview,” available at <http://issat.dcaf.ch/Learn/SSR-Overview>.

5 This is the assumption behind the Ministry of Defense Advisors Program.

6 See Merilee Grindle, “Good Enough Governance: Povery Reduction and Reform in Developing Countries,” Governance: An International Journal of Policy, Administration, and Institutions 17, No. 4 (2004), 525-548.

7 Although practitioners in other contexts are often careful to refer to “good practices” to signal their adherence to the principle of “best fit, not best practices,” DIB practitioners sometimes continue to refer to “tailored international best practices (IBP)” for a number of reasons. First, given DOD past practices, the reference to IBP often serves as a shorthand way to communicate to both U.S. and partner-nation stakeholders that the advisor is not promoting U.S. mirror-image solutions. Second, the term still holds a powerful aspirational appeal in many settings. Reformers may be dissatisfied with the status quo and want change, but are not entirely sure what that entails; therefore, the initial request is often to understand IBP. Finally, being able to say that proposed new practices are international standards can also be an effective weapon for reformers. The danger of IBP comes not in the use of the term, but when IBP are converted into models and used by practitioners—or policy makers—as a blueprint for action.

8 Lisa Denney, “Using political economy analysis in conflict, security and justice programmes,” March 2016, available at <https://www.odi.org/sites/odi.org.uk/files/resource-documents/10362.pdf>.

9 World Bank, World Bank’s Approach to Public Sector Management 2011-2020: Better Results from Public Sector Institutions (Washington, DC: World Bank, February 2012) available at <http://siteresources.worldbank.org/EXTGOVANTICORR/Resources/3035863-1285601351606/PSM-Approach.pdf>, 10.

10 For the sailboat analogy, see Rachel Kleinfeld, “Improving Development Aid Design and Evaluation: Plan for Sailboats, Not Trains,” March 2015, available at <http://carnegieendowment.org/2015/03/02/improving-development-aid-design-and-evaluation-plan-for-sailboats-not-trains/i4u1>. This less linear view of change does not preclude systematic thinking about problem-driven sequencing of reform efforts to expand the space for reform. See Matt Andrews, Lant Pritchett, and Michael Woolcock, “Doing Problem Driven Work,” Center for International Development at Harvard Working Paper No. 307, December 2015, available at <http://bsc.cid.harvard.edu/files/bsc/files/doing_problem_driven_work_wp_307.pdf>.

11 International Security Sector Advisory Team, “Operational Guidance Note: Overview for Security and Justice Assessment,” April 2010, available at <http://issat.dcaf.ch/content/download/1176/8908/file/ISSAT%20Assessment%20OGN%200%20Overview.pdf>, 6. The Marine Corps Security Cooperation Group came to a similar conclusion in their efforts to assess partner-nation Marine Corps as a first step in proposing a plan of engagement: a comprehensive DOTMLPF assessment was usually not feasible, nor advisable.

12 “The World Bank’s Approach to Public Sector Management,” op. cit., 12. For problem-driven iterative adaptation, see Matt Andrews, The Limits of Institutional Reform in Development: Changing Rules for Realistic Solutions (Cambridge: Cambridge University Press, 2013).

13 International Peace Institute, “Security-Sector Reform Applied: Nine Ways to Move from Policy to Implementation,” February 2012, available at <https://www.ipinst.org/2012/02/security-sector-reform-applied-nine-ways-to-move-from-policy-to-implementation>.

14 The use of the term “defense institution building” within the U.S. Department of Defense comes from the WIF-NATO experience and has stuck, despite the challenges it poses when interacting with partners, who rightly note that they already have institutions and do not need them to be “built.”

15 Some policymakers use the term “responsible civilian control” instead of “democratic civilian control” to capture the idea that the military needs to be transparent and accountable even in less-than-democratic settings, as well as to indicate that civilian control must follow some standard principles (like transparency) in a democracy.

16 See Office of the Secretary of Defense for Policy, DOD Directive 5205.82 Defense Institution Building (Washington, DC: U.S. Department of Defense, January 27, 2016), 13. DIB programs also routinely support ministries of defense as they interact with actors external to the defense sector, like the Ministry of Finance and Ministry of National Planning.

17 DoD-Directive 5205.82 Defense Institution Building, op. cit.

18 Similarly, the Marine Corps Security Cooperation Group replaced their initial model of comprehensive DOTMLPF-P assessments with a “scoping” approach based on U.S. and partner-nation objectives that ensured the requirements for information and action were realistic and achievable.

19 Christopher Paul et al., What Works Best When Building Partner Capacity and Under What Circumstances? (Santa Monica, CA: RAND Corporation, 2013), available at <http://www.rand.org/content/dam/rand/pubs/monographs/MG1200/MG1253z1/RAND_MG1253z1.pdf>.

20 One example of breaking down stovepipes between stages of the security cooperation planning process is provided by the Marine Corps Security Cooperation Group, which has adopted the good practice of including an individual from the assessment team that conducts security cooperation planning on the teams that subsequently deploy to execute the planned activities. In this way, knowledge gained of country context and the intent of the activities is more effectively conveyed to implementers than is normally the case in security cooperation. In another example, Special Operations Command Africa has also effectively employed the good practice of combining the planning and evaluation personnel to ensure that plans and objectives are designed with clearly defined outcomes in mind.

21 Thomas W. Ross, “Enhancing Security Cooperation Effectiveness: A Model for Capability Package Planning,” Joint Force Quarterly 80 (January 2016).

22 The White House, Presidential Policy Directive 23: Security Sector Assistance (Washington, DC: The White House, April 5, 2013), 12.