News | Nov. 8, 2018

The Mandate to Innovate

By Ms. Christina Monaco PRISM Volume 7 no. 4

Download PDF
Ms. Christina Monaco is the Chief Innovation Officer at the National Geospatial-Intelligence Agency.

For years the U.S. Intelligence Community (IC) has been grappling with a threat environment that is growing exponentially more complex. The U.S. National Security Strategy and National Defense Strategy both emphasize the need to rapidly develop and integrate innovative technology. It is no longer enough for intelligence professionals to focus on delivering on their operational missions. We must also identify the means to continuously replenish the nation’s intelligence advantages to match current and future threats. In other words, we have a mandate to innovate.

In autumn of last year the new Office of Ventures and Innovation (OVI) officially stood up, with a mission to guide partnerships, pathways, and solutions to improve upon existing, invent new, and imagine transformational geospatial-intelligence (GEOINT) capabilities.1 In a move somewhat “meta-innovative,” OVI includes not only functions of the activities that are more conventionally included in similar offices (including sponsoring of hackathons, challenges, and prototyping using Other Transactions Authority, or “OTA”)—we also include NGA’s acquisition governance activities as well. This reflects Agency leaders’ understanding that in order to innovate successfully, we must take into account the full spectrum of the acquisition lifecycle. Thus, OVI helps NGA source innovation, incubate potential solutions, adopt new capabilities, and mainstream successful capabilities in programs of record.

New Business Models

OVI is a relatively new organization at NGA, but we are not unique. Counterpart organizations exist in many other federal government and Department of Defense (DOD) entities. While the U.S. national security community has a long history of innovation, the establishment of organizations specifically dedicated to innovation within the government is becoming a norm. Taking a cue from such successful organizations as the Advanced Research Projects Agency (ARPA) that was established in 1958 by President Dwight D. Eisenhower, new DOD and IC organizations have been established in recent years. The Defense Innovation Unit (DIU), the Defense Digital Service, SOFWERX, and CIA’s Directorate of Digital Innovation were all established in 2015, and other similar organizations have followed suit. The impetus is primarily the recognition that it is a “disrupt, or be disrupted” world out there. The status quo for innovation and acquisition of new capabilities will be akin to running in place, as noted by Steve Blank in his article “The Red Queen Problem: Innovation in the DOD and Intelligence Community.”2 New organizational and business models are therefore necessary to ensure we can outpace our adversaries’ efforts to modernize. Throughout NGA—and particularly in OVI—experimentation with such models is gaining traction. In some cases, entirely new concepts are being tried. And in other cases, we are finding ways to improve upon existing practices.


Growing in popularity throughout DOD, Other Transaction Authority agreements, or “OTAs,” allow for the rapid acquisition of research and prototype capabilities. OTAs are not subject to the Federal Acquisition Regulation, can be up to $250 million, and either all commercial participants are small businesses or non-traditional vendors, or a non-federal government agency funds at least one third of the total cost of the project. Within DOD, OTAs are intended primarily to support acquisition from “non-traditional” contractors, defined as entities not currently performing (or that have not performed for at least one year prior to the solicitation) either a contract or subcontract subject to full coverage under the Cost Accounting Standards. Traditional vendors can, however, be awarded OTAs if they partner with non-traditional organizations to a significant extent.

If an OTA is competitively awarded, any post–prototype capability acquisition may be sole-source awarded to that vendor. Alternatively, an OTA may be non-competitively awarded, but post–prototype acquisition would be subject to the normal competitive process. Many organizations (including NGA) award OTAs through capability-oriented consortia formed by industry partners of all sizes. Others, such as DIU, leverage commercial solution openings.

NGA recently awarded an OTA agreement through a consortium to a vendor that had never before directly contracted to NGA, and is the epitome of a small business: a sole proprietor offering uniquely developed cognitive artificial intelligence algorithms, software applications, and application programming interfaces (APIs) to automatically fuse and validate multiple streams of contextual geospatial intelligence data—that is increasing in volume exponentially—into NGA’s analysis. One of the challenges NGA’s analysts find is that they need to create and maintain highly customized analytic models for specific intelligence problems, but their analytic outputs must conform to a set of enterprise standards in order to be searchable and interoperable in a practice referred to as “structured observation management.” Normally the translation of the customized models to standard form outputs would be highly manual drudgery for NGA analysts whose time and attention are better spent solving intelligence problems. The prototype, if successful, will reformulate imagery data collected in the diverse analytic schemas into the required outputs. The NGA “problem owner,” an analyst named Jeremy Boss, noted:

The simplicity and adaptability of the OTA make it a key resource as we pursue automation at NGA. The simplicity of the process makes it accessible to analysts without an acquisition background, and its high degree of adaptability makes it customizable to analysts’ current challenges. Together, what this means is that the people closest to analytic problems are able to design innovative responses, greatly improving problem-solution fit.


NGA is also expanding use of Cooperative Research and Development Agreements (CRADAs) with non-traditional partners, such as start-ups and companies whose business is not primarily related to geospatial technologies, but with which NGA has some shared interests. While NGA’s CRADAs do not involve the exchange of funds with CRADA partners, several benefits accrue to these entities. These primarily relate to cost avoidance and access.

For a small business, research and development (R&D) can be a very costly undertaking, financed either through debt or equity. Capital financed through debt may be difficult or prohibitively expensive, and capital raised through equity financing may force a founder to relinquish more equity (and corporate control) to outside investors than they would like, particularly in early stages of start-up. Cost-sharing of R&D with a stable, safe (particularly in terms of intellectual property) partner like the federal government offers an attractive option. Firms involved in CRADAs with NGA have access to a ready pool of expert and trustworthy co-developers and customers. In many cases, partners can also gain access to unique geospatial data, imagery, storage, and compute resources for which they would otherwise have to pay. In turn, NGA gains insights into novel research or early access to developing and cutting-edge technologies.

Small businesses may also accrue other benefits through CRADAs. Such agreements can be used, for example, to justify processing of facility (and personnel) clearances. Companies may also note past experience with the government as CRADA partners as they position themselves for future procurement actions. Finally, CRADAs have the intangible benefit of demonstrating market interest in a capability—this becomes important during a firm’s efforts to raise capital.

Hackathons and Challenges

These days, it seems you cannot be considered an innovative organization without holding a hackathon or an incentived prize and challenge competition. At NGA, we have held several of these. And while such events and contests can yield interesting insights about novel approaches, identify unknown opportunities (or vulnerabilities), and build goodwill and outreach, often the promise of our efforts has remained largely unrealized. In fact, hackathons and challenges have something of a rap for being “innovation theater,” or events you must hold to seem open to innovation, but which do not actually result in the adoption of new solutions.

A few organizations, however, have cracked the code to increase the return on investment. To wit, the fine print that accompanies SOFWERX challenge announcements reads:

Announcement of TeamWERX prize challenges, sponsored by SOFWERX, that are considered to have high potential for further efforts that may be accomplished via FAR based contracting instruments, Other Transaction Authority (OTA) for Prototype Projects 10 USC 2371b, Prizes for advanced technology achievements 10 USC 2374a, and/or Prize Competitions 15 USC 3719, may be made at the and website. All announcements made at the website(s) are considered to satisfy the reasonable effort to obtain competition in accordance with 10 USC 2374a (b), 15 USC 3719 (e) and 10 USC 2371b (b)(2). All FAR based actions will follow announcement procedures per FAR 5.201(b) accordingly.3

In other words, SOFWERX allows challenges to be used as the competitive element necessary to award a subsequent contract action. The specific clause that permits this in the Title 10 section that applies, S.2374a “Prizes for advanced technology achievements,” is subsection (d), which reads:

Relationship to Other Authority: A program under subsection (a) may be carried out in conjunction with or in addition to the exercise of any other authority of an official referred to in that subsection to acquire, support, or stimulate basic, advanced and applied research, technology development, or prototype projects.

These authorities hold much promise for turning innovation theater into “innovation reality shows.” This concept is also at the base of one of OVI’s major efforts, GEOWorks.

GEOWorks and Public Private Partnerships

The GEOINT discipline has evolved rapidly from pictures to pixels to data. To make sense of the flood of data now streaming into the IC and DOD from both government and commercial GEOINT capabilities, we need to develop machine learning and computer vision technologies. The advanced technologies we need are not always rapidly available to the government. To ameliorate this, in 2017, NGA rolled out a new concept for a means of public-private partnership (PPP) that leverages our existing holdings of petabytes of geospatial data and imagery for the development of technologies related to what we refer to as “AAA,” or automation, augmentation, and AI. Our value hypothesis is that by providing outside entities access to NGA’s data, the partners could use them, for example, as training datasets for the development of AI algorithms with potential for both government and commercial purposes. By partnering with NGA in this fashion, new AAA technologies might be invented, and NGA could benefit not only from the availability of these technologies, but also by receiving licenses and government use rights for the co-development efforts.

In true start-up style, NGA has begun a number of experimental means of testing the hypothesis that there is a market for these kinds of geospatial data partnerships. First, we have been pursuing CRADAs with companies willing and able to experiment alongside NGA on this effort. What we have initially found is that there is broad interest in co-developing technology in this manner, yet we must carefully consider risks and benefits when it comes to the intellectual property rights that would accrue as a result of such efforts.

Second, we created a new website that allows for open access to NGA data holdings and some basic processing and compute resources. This website, “GEOWorks,” ( was developed and launched in just eight weeks. The website leverages for user access control, allowing the public to access several NGA and other publicly available geospatial datasets. At this point, the data is largely foundation data (related to mapping, charting, and geodesy), including for example terrain elevation data, vertical obstructions, and navigational information—and we are pursuing the release of additional unclassified or declassified data (such as historical imagery from satellites). While the data cannot be downloaded by users, we are also providing access to various processing capabilities, such as a geospatial viewer, TensorFlow, and Jupyter Notebooks. In order to maximize interest in the data and the GEOWorks platform—and eventually to allow for GEOWorks to be a platform from which NGA directly acquires new technology (e.g., software or AI algorithms)—the site includes links to ongoing hackathons and challenges. Our intent is to use this latter capability to identify the most promising AAA technologies and formally acquire them through, for example, OTAs leveraging the authorities cited above. The hackathons and challenges to be posted will be actual AAA needs to which the datasets posted on GEOWorks relate. And while GEOWorks is a technical underpinning for our PPP efforts, the site also benefits NGA for purposes such as recruiting and outreach. Academia will be able to use the data and capabilities available on GEOWorks to practice geospatial data science and analysis, enriching the talent stream that feeds NGA and the GEOINT enterprise.

GEOWorks has some indirect benefits to NGA in that its development and launch have helped the agency adopt product management practices. We have leveraged our recent successes with NGA’s GEOINT Services efforts, and are applying the “build low, move high” mentality in which we first test and prove capabilities on unclassified networks before we integrate them into our classified networks and workflows. For example, one of the subcomponents of GEOWorks is figuring out the technical aspects of ingest and integration of algorithms developed through the platform. As a result of this learning aspect of the project, we are better able to integrate agile, lean, and iterative technology practices.

Acquisition Restructuring:
Organizing for Agility

Innovation at NGA is not just about new capabilities. We are also focusing on innovation in organizational structures and processes to achieve the elusive goal of agility. One of the first activities undertaken by the NGA Office of Ventures and Innovation was to overhaul acquisition oversight within the agency. NGA Director Robert Cardillo recently designated his deputy, Justin Poole, as the agency’s Component Acquisition Executive (CAE). For several years, that authority had rested deeper within NGA’s management structure. However, as part of a recent restructuring of senior management at NGA, Director Cardillo felt it prudent to elevate the CAE role. In order to ensure not only that acquisition oversight was being properly managed, but also that innovation is actively incorporated into NGA’s programs, the Deputy CAE and staff support for that function aligns to the OVI.

In partnership with NGA’s Chief Information Officer (CIO) and Information Technology (IT) management directorate leadership, Poole has implemented a new acquisition governance schema in his role as the CAE. First, these leaders agreed on a rough number of what we have come to know as “big chunky programs” organized by GEOINT mission outcomes, such as “sensor integration” or “foundation GEOINT modernization.” (Prior to this designation, NGA had dozens of programs, mostly aligned to existing workflows that in turn were reflected in singularly-aligned contracts). The consolidation of programs aligned to mission outcomes is allowing program managers to have a more holistic view of all the related capabilities in their portfolios. They can more easily identify potentially duplicative capabilities, prioritize development backlogs, and manage budgets with greater accountability.

These big chunky programs and their program managers are adopting modular contracting strategies, looking to integrate in-house software development and DevOps efforts, and participating in technology and innovation boards that offer the opportunity to introduce technologies to satisfy requirements in their backlog or to potentially offer new capabilities of which the customer was unaware. To ensure that the aggregation of existing programs and requirements did not become a bureaucratic roadblock, the CAE has identified and delegated decision authority for program and service acquisition approval/oversight to the agency’s Senior Procurement Executive and to the CIO, consistent with their statutory responsibilities. This is a subtle but meaningful cultural shift in an agency which previously required any program or contract action to be reviewed and approved by the CAE when the dollar threshold was greater than $35M.

Furthering the cultural change, NGA’s CIO and IT management directorate is leading the creation of new NGA Integrated Program Offices that align to the “big chunky programs.” These integrated program offices are being designed around the need to have customers and subject matter experts (mission owners, developers, security personnel, contracting officers, and innovation advisors, to name a few) working daily with efforts that are employing agile software development methods or DevOps. By fully integrating these stakeholders into the program offices and tying mission and product owners to the development side of the house, NGA is organizing for rapid delivery and acceptance of mission enhancing capability to the customer. The kind of tangible outcome that will result from this structure is, for example, the broader adoption of NGA’s “ATO-in-a-Day” (authorization to operate) assessment and authorization process for new software.

Not all innovative ideas come from inside NGA. As an example, NGA also awards academic grants that support innovative, high-payoff research that provides the basis for revolutionary progress in areas of science and technology affecting the needs and mission of the Agency. (NGA)
Not all innovative ideas come from inside NGA. As an example, NGA also awards academic grants that support innovative, high-payoff research that provides the basis for revolutionary progress in areas of science and technology affecting the needs and mission of the Agency. (NGA)
Not all innovative ideas come from inside NGA. As an example, NGA also awards academic grants that support innovative, high-payoff research that provides the basis for revolutionary progress in areas of science and technology affecting the needs and mission of the Agency. (NGA)
Photo By: NGA
VIRIN: 181104-D-BD104-020


All new business models must be supported by an organizational culture that thrives on experimentation and learning and is willing to “fail forward.” As noted by the U.S. House of Representatives Armed Services Committee Chairman Mac Thornberry (R-Texas) in 2016, experimentation:

encourages innovative thinking, not just in developing the technology, but in how you use it. It helps ensure there is mature technology before you start production so that you don’t have those unexpected surprises. It reduces the odds that you are going to spend a lot of money on a program of record that you then have to cancel and have it all wasted.4

In other words, there is room in the federal government to adopt some of the much-ballyhooed fail fast, fail often mantra of Silicon Valley. To do so—even in government—is a realist’s way of avoiding the far worse infraction of failing late, and failing expensive. In fact, in Silicon Valley and the D.C. Beltway, the worst crime might be not to try at all. At NGA and in other partner organizations, we are encouraging entrepreneurial practices similar to those practiced in the start-up world in order to grow more comfortable operating with this mentality.

Lean Start-Up

Lean Start-Up practices as made popular by Eric Ries and widely adopted in the business world have a foothold in the government as well. Within the IC, several agencies (including foreign ally agencies) have joined efforts in an education program called “Fast Forward” that is training intelligence professionals from virtually all disciplines. The program particularly emphasizes problem curation—the effort that must be put into talking to customers and other stakeholders and in surveying the existing landscape of possible solutions to ensure capabilities being pursued are truly needed and innovative. Fast Forward participants come away from the course with a “mission model canvas” that kick starts their efforts to solve a particular problem and create value for beneficiaries. Using a concept outlined by entrepreneurs Alex Osterwalder and Steve Blank, this artifact helps innovators visualize how to turn customer needs into mission outcomes.5

At NGA, we have also formalized a six-phase “lean innovation pipeline” model based on the work of Blank, Osterwalder, and Pete Newell of BMNT (and former director of the Army’s Rapid Equipping Fund). The model aligns with NGA’s existing corporate and acquisition governance processes, to provide a roadmap for moving projects from concept development through integration. One of the needs OVI has identified from past and ongoing innovation efforts at NGA is that agency innovators need a clearer understanding of the means available to them and the potential pathways they could take to move their projects through this pipeline. To address this, OVI developed an internal web-based reference to help agency innovators identify the best means available to advance their innovation projects through the phases of innovation, from sourcing, to problem curation and prioritization, into solution/hypothesis testing, incubation, and finally transitioning to integration. In OVI we also have established a cadre of experts in the pipeline and in lean start-up practices who serve as “caddies” working alongside agency innovators to advise them as they move through these processes. This “full service” assistance model is particularly useful for first-time innovators, many of whom seek our assistance as a result of participation in NGA’s formal innovation contest.

Innovation Experience

One means NGA has implemented to both identify needed capabilities and to nurture innovative business practices from within the agency has been something we call the “NGA Innovation Experience.” Similar to programs such as the Spark Program sponsored by the Director of National Intelligence, what sets the NGA Innovation Experience apart is how we bring together concepts from the start-up community and venture capital world in a government context. The program solicits ideas for innovation projects from the NGA workforce, and teams interested in participating can take NGA’s Fast Forward courses. There, they become familiar with the concepts of lean start-up and product/market fit. After an initial round of judging by executives from across the agency, the teams participate in a showcase to pitch their ideas to NGA’s Venture Board. The board includes a smaller subset of executives, including C-Suite leaders, who have the authority to grant some teams the time to step away from their day-to-day missions for a month of intensive innovation accelerator activity. Similar to commercial start-up incubators, the accelerator staff help the teams to refine their problem statements, develop “minimum viable products” to better understand customer needs, and develop solution hypotheses.

After this month of activity, the teams return to the Venture Board to make another round of pitches, using data collected during their accelerator participation and investment-based logic to convince the Board to further advance their ventures. In this round, the Board can award either additional on-the-job time or the needed resources from a Venture Fund set aside for these types of projects. This concept of incremental funding based on data proving projects’ alignment to key “-ilities” (such as viability, feasibility, and desirability) is specifically what helps the Venture Board avoid locking up agency resources in solution investments that have little initial proof of future success. This approach helps the agency buy down technical, cost, and even schedule risk. It also puts the user at the center of both the product design and resourcing decisionmaking processes.

In some cases, the teams participating in the Experience accelerator find that they must pivot off their initial concepts in order to better solve the problem they have identified. In 2017, one such team, known as Team MEANS (Modern Environment for Analysis of Networks), entered NGA’s accelerator with full intentions to build a tool for road network analysis to support NGA military customers that would have integrated existing road networks databases (akin to Waze for tanks). After spending time curating their problem and conducting customer discovery, they realized that without underlying data hierarchical integration their original concept wouldn’t work. As a result, the team pivoted to creating this hierarchy, prior to building out any routing analysis algorithms. Absent this problem curation, the solution Team MEANS developed would not have the “-ilities” necessary to successfully achieve the intended outcome, and would have been a waste of time and resources.

Of course, in the world of geospatial intelligence, not all innovative ideas for NGA’s mission come from inside the agency. As the geospatial industry grows and expands to include such new technologies as artificial intelligence, machine learning, and computer vision, we at NGA are finding we need new relationships with non-traditional partners—many of whom do not see DOD or the IC as customers. NGA has established small teams we call “Outposts” in Silicon Valley and the Austin/San Antonio, Texas corridor to identify such new potential partners and scout technologies that could help advance NGA’s capabilities. The Outposts are staffed with personnel representing specific NGA missions (e.g., intelligence analysis, mapping/charting/geodesy, geospatial analysis, security, IT) who fill the role of translating NGA’s needs into commercial language and vice versa. When they find a technology with unique potential to fill mission needs, the Outposts are able to leverage a budget of “seedling” funding to place on OTAs or to negotiate CRADAs. By virtue of their locations, Outpost personnel are able to build deep professional networks in the country’s cradles of innovation. In addition to direct engagement with innovative commercial solutions providers, the Outposts are building relationships with venture capital firms and start-up incubators. These parts of the innovation ecosystem are often in a position to guide the corporate and product maturation for technologies and services that could be of use for GEOINT missions, and to connect NGA with innovative new solutions.

The Outposts often set up “technical terrain walks” for NGA personnel who are involved in problem curation, so they can better understand what commercial solutions have already been developed. Recently, the Silicon Valley Outpost hosted a small group visiting from NGA’s headquarters in Springfield, Virginia and from our St. Louis, Missouri location who were curating needs related to NGA’s role as the IC/DOD functional manager for GEOINT. Specifically, this group was beginning an effort to completely revamp how the GEOINT enterprise requests capabilities needs and how those needs are tracked until they are delivered. The group engaged with a number of companies that have grappled with similar issues, such as Sevanta DealFlow, Appian, SurveyMonkey Apply, and Composable Analytics. The participants came away from the visit with a better sense of how analogous organizations tackled these problems. After their terrain walk, the group convened a larger group in Springfield to design a capabilities requirements management process for the GEOINT enterprise, leveraging what they learned in the Valley, and is on track to deliver a new process this fall.

The Outposts also serve as talent scouts for the agency, bringing attention to the possibility of federal service in tech hubs rich with talent. And just as importantly, the Outposts serve as “culture scouts.” Outpost personnel, working alongside partners in industry, academia, and government help identify new methods and practices that NGA can apply to improve the acquisition of new capabilities. Additionally, the Outpost teams are proving that not only is it possible for an intelligence agency to operate in a largely unclassified environment, but there is great value in doing so. Using OTAs we can quickly bring in commercial technologies, assess their “-ilities,” and move to integration in a fraction of the time it would take to pursue a more traditional procurement.

What Can We Improve?

While NGA’s efforts to maintain an innovative GEOINT advantage are advancing, there are some areas still ripe for corporate innovation. The Agency’s agile innovation backlog still includes, for example, the following requirements.

Tech Insertion

Program managers have long been encouraged to plan for tech insertion as they build out their programs of record across the future years defense program (FYDP). Yet, as budgets tighten, it is often tempting to cut such funds from plans. After all, when every dollar is scrutinized by oversight, loosely defined tech insertion “wedges” without a lot of detail on specific capability are tempting targets for realignment to other needs. Yet a program without funding for tech insertion is a broken and wasteful program in the making—practically guaranteeing that by the time of delivery, the technology will be obsolete. We must vigilantly plan for and protect tech insertion funds, similar to how we protect R&D through program guidance. In fact, this need may even warrant a new “color” of appropriation—or even “colorless” funding that can be applied in the year of execution to any appropriation, as long as the funds are specifically applied to acquisition of technology that was developed inside the window of the program cycle.

Workforce Development

Rapid innovation requires the acquisition workforce to be conversant in both technology and the models being applied in the commercial world. Exposure to fundamental concepts of software development/coding and data science will not only help the workforce better appreciate and understand the programs they manage, but will also develop skills in critical and computational thinking. Additionally, widespread adoption of coursework in business concepts such as agile software development and lean start-up will provide insights into commercial practices and help the workforce become comfortable with the “fail fast” mentality that is needed to succeed as innovators and entrepreneurs. The Defense Acquisition University has piloted the “Hacking for Defense” course, and the U.S. House of Representatives has included a related amendment to the National Defense Authorization Act that requests DOD to explore expansion opportunities for this program. These are tremendously encouraging developments, given the reach of professional education throughout DOD. Beyond training, managers must encourage and reward the workforce as they engage with industry and academia to maintain insights about the current and future state of technology. Only by occasionally stepping away from the day-to-day rigors of their jobs will an empowered workforce have the strategic technological perspective needed to inform their innovation efforts.

At the more senior levels, defense and intelligence executives also need to understand how to apply decisionmaking criteria in a lean start-up context. When deciding whether to allow an innovation project to begin, what kinds of data should executives operating with a venture capital mindset expect to see? (Hint: Successful angel investors listen for clues about unmet market demand or creative ways around traditionally high barriers to market entry in this phase.) And how can defense and intelligence executives decide whether to provide more time or resources to a project that has developed a solution to a hypothesis? (Another hint: look for actual customer data indicative of product-market fit.) At NGA, we offer a course specifically tailored to executive decisionmakers to introduce them to these concepts. This course focuses on managing uncertainty, strategic choice making, innovation at scale, and innovation accounting and metrics. As leaders become more comfortable with applying these business concepts to our mission, we should be able to more rapidly and efficiently support innovation and acquisition throughout the agency.

Innovation Metrics

As Alex Osterwalder has noted, many organizations struggle with measuring innovation:

How you measure results for a known and proven business model or value proposition substantially differs from how you measure progress in an innovation project for an unproven potential business model.6

He goes on to note that the application of traditional execution metrics to innovation will doom such projects to failure. But how are we to know if government innovation efforts are worthwhile investments of time and taxpayer dollars?

First, we must recognize that not every innovation project will be successfully integrated. However, every innovation project will result in organizational learning. We can begin to measure learning as an additional return on time and resources spent on a project. One way to do so in a government context is to establish up front a set amount of time that will be spent on an innovation project, and to track the use of that time similar to how a start-up would track the burn rate on their available capital. At the same time, the innovation project should define a set of learning objectives it will accomplish. Tom Chi, formerly of Google X, notes that tracking key learning, which he defines as “an embodied or observed experience that materially changes the path forward,” is an entirely desired outcome for any learning organization.7 The kinds of learning objectives to be tracked could include, for example, identifying the means to reduce risk or increase the impact of a project. Chi has also spoken of not only metering time to innovation projects, but also establishing a rigorous schedule for prototype development for those projects (e.g., we will spend four months and build exactly 100 separate prototypes). By setting and adhering to that kind of goal as a condition for further investment decisionmaking, teams are incentivized both to move at pace and to accomplish as much learning as possible. When that time is up, the organization decides whether or not to further invest—and that decision should include a review of whether the learning objectives set out up front have been met, and an assessment of the value of any further learning that could accrue with the project. Similar rigor could certainly be applied in a government context.

Additionally, at the corporate level, the CAE should conduct a periodic accounting of the percent of programs with specific innovation efforts underway, as well as the percent of innovations being applied to each of three future horizons. Based on a model defined by McKinsey & Company, in NGA’s OVI we define these horizons as improving existing products and services, inventing new ways to accomplish existing lines of business, and imagining entirely new and transformational lines of business.8 While tracking such metrics would not necessarily tell us about the effectiveness of the innovation efforts, it would give us a sense of the scope of innovation across the organization.


At the end of the day, all of NGA’s efforts to innovate relate to one singular purpose: delivering GEOINT to our customers when and how they need it. It is certainly easy for us to become endlessly fascinated by new capabilities and wrapped up in the minutiae of corporate processes. Yet delivering decision advantage to a policymaker or situational awareness to a warfighter is becoming a more competitive challenge. As geospatial and AAA technologies increase in capability and availability—both within the United States and allied GEOINT enterprise as well as for our adversaries—the complexity of that mission increases. Empowering rapid experimentation and innovation, adopting new business models (particularly those that have proven successful in the business world), and applying the breadth of the means available to us to acquire new capabilities are ways for us to continuously replenish the nation’s GEOINT advantage. PRISM


1 Steve Blank, “The Red Queen Problem, Innovation in the DOD and Intelligence Community,” SteveBlank (blog), October 17, 2017, available at <>.

2 SOFWERX challenge announcements are available at <>

3 Scott Maucione, “Mac Thornberry Wants DOD to Start Failing,” Federal News Radio, January 13, 2016, available at <>.

4 Steve Blank “The Mission Model Canvas—An Adaptive Business Model Canavas for Mission-Driven Organizations,” SteveBlank (blog), February 23, 2016, available at <>.

5 Alexander Osterwalder, “Innovation Metrics vs. Execution Metrics,” Strategyzer (blog), November 20, 2017, available at <>.

6 Tom Chi, “Rapid Prototyping & Product Management by Tom Chi at Mind the Product in San Fransisco,” Mind The Product, posted toYouTube on May 31, 2018, available at <>.

7 “Enduring Ideas: The Three Horizons of Growth,” McKinsey Quarterly, December 2009, available at <>.