NPS Badge

U.S. Naval Postgraduate School

The Data Collection and Management Plan

Example of a Data Collection and ManagementA Data Collection and Management Plan (DCMP) is developed with issues decomposed into sub-issues, and further, perhaps, into questions. The decomposition may continue through essential elements of analysis (EEA) and the measures of merit (MOM) that might be used to assess them. If appropriate, a DCMP can be decomposed to a very fine granularity, usually called a data element (DE).

The DCMP may also be used to develop an inventory of information that needs to be provided to players so they can fulfill their roles. Peter Perla [The Art of Wargaming, p. 166] explains: "The data base contains the information players may use to help them make decisions. Typically, this information includes the forces available, some measure of their capabilities, physical or environmental conditions, and other technical facts. Because of its importance to decision making, the data base must present clearly and concisely the information players would reasonably have available to them in an actual situation, and it must do so in a manner easy for them to use during play."

The data that is provided to players may often be extracted from a combat simulation. In particular, a combat simulation may provide players with the current situation of their own forces -- the forces that remain combat effective, their location on the battlefield. Combat simulations are also effective in providing players with information on their adversaries, based on detection algorithms within the simulation.

Simulations may also be useful in tracking other information that may be provided to players, for example simulations of consumption of supplies, the abilities of the logistics organization in replenishment, or the combat effectiveness of personnel casualties.

In simpler seminar war games where such simulations may not be employed, staff supporting the game may have to track such information. This might be to indicate what forces a player has remaining and the state of their morale.

For seminar war games, where data collection may be limited largely to a record of the discourse between players, the MOMs would not often be quantitative measures. Data collection is more likely an assessment of qualitative views or of military judgment. Even though most seminar war games would require a far simpler DCMP than seen in many other forms of game, a DCMP is still valuable in designing a scenario that will hit on the issues that emerge from discussions with the sponsoring organization. During the conduct of the seminar war game, the DCMP provides a useful checklist to ensure that no relevant issues have been overlooked; using a DCMP should ensure that the study team will avoid reaching the end of a game only to discover that data on some important issue was never collected.

Spreadsheet formatting for a DCMPGenerally a DCMP is developed in a spreadsheet, as a convenient and familiar tool. Apart from the contents already indicated, columns can be added for many more elements.

Across a row in a DCMP spreadsheet, one should find an indication of the issue to be addressed (derived from the sponsor's objective). If an issue is particularly complicated, it may be decomposed into a number of sub-issues. Then further to the right of the issues/sub-issues will be the essential element of analysis (EEA). And to the right of this will be a measure of merit (or perhaps several with a row for each). Note: there may be more than one EEA for each sub-issue, and more than one MOM for an EEA. For a MOM, there may be several data elements (DE), if the decomposition is to that level of details.

For example, an EEA may deal with casualties, which could result in a collection of several MOMs or DEs. It may be necessary to collect BLUE and RED casualties, and these casualties might be sub-divided by personnel and vehicles. And there could be further categories for BLUE-on-BLUE casualties or collateral casualties to the civilians population.

While the obvious purpose of a DCMP is to develop an inventory of expected data (at the levels of MOM and DE), by extending the DCMP to the right, the DCMP encourages the development of elements of the scenario -- clearly no data can be collected for an issue/sub-issue if there is not a point in the scenario where the players are confronted by it. To extend the previous example, no data will be collected on casualties unless something in the scenario creates the potential for casualties. And further, if collateral casualties to a civilian population is a concern, there has to be potential within the scenario for such casualties to occur.

Player Information

Phased Approach As issues and sub-issues are recorded in a DCMP, it should be clear what sort of information players will need to deal with these as they play the game. One or more columns in the DCMP spreadsheet can be allocated to this. Some obvious types of information will be the status of own forces and those of other sides (and especially an enemy). Some that may be less obvious would be for logistics: "do we have enough supplies?", "are they in the right places?", "can they be delivered in time?" Or they may be about the local population: "Are the locals accepting of our presence?", "Are any of our activities likely to antagonize them?" Some aspects of context, especially intangibles might be the consequence of myths or biases that developed in players long before they came to the game; so data collectors must be sensitive to this.

Much of the information provided to the players as the game unfolds will be dynamic. The table of organization and equipment (or order of battle) provided at the start of a game needs to be modified in light of combat results and some other effects. For some types of games, players themselves might track losses and other types of games data might be provided to represent remaining forces. Most other data will change too: in a way, players make their decisions in order to affect the data that other players have (although players would not see their moves in such clinical terms). Players are generally trying to deplete the resources of their opponents (which, for decision making, is provided to those opponents as "input data".

When a list of the player information has been compiled (from appropriate columns in a DCMP), this provides a "checklist" of sorts for Step 5 "Search for Information" in the phased approach for a gaming project.

Data to Be Collected

For game analysts, the component of a DCMP that is at the heart of their role covers the data to be collected. In a way, this is a "wish list" for what the analyst's hope to have once the game has been completed. Of course, "hope is not a method" [according to GEN Sullivan's book]. So this part of the DCMP is actually much more than a "wish list": it should provide a framework for how the game is designed and developed. The required data is based on the sponsor's issues (derived from the objective(s)). If data is not collected on some issue, then that aspect of the sponsor's expectations will be unmet.

Data on Context

When players make a decision, it will be within a certain context. The context is certainly likely to depend on factors like the capabilities of one's own forces. It may depend on factors that are less tangible, such as quality of training. And very intangible factors might enter into the decision making, for example, expected responses from the local population or an assessment of morale, both of own forces and other forces.

Some of the intangible aspects of context may be based on shared experiences of a player group. For example, a staff that has not been exposed to critical cultural aspects of the local population may make certain assumptions about that population's behaviour that may not be well founded. Apart from the players, the analysts and data collectors may also be oblivious to some intangibles, so considerable care must be taken to assess such intangibles.

Data from Models, Methods, and Tools (MMT)

Various MMT may be incorporated into game play. In particular MMT may be used in the adjudication process, determining how far an entity might move in a given period of time, whether they can be detected, what they can detect, what their levels of supply and other resources may be, whether they engage with targets or are engaged. Appropriate data should be recorded for all such events.

Games that focus on command and control should have a means of recording the effectiveness of underlying communications networks, e.g., available bandwidth, latency of messages. As with the results from other MMT, a record of appropriate statistics needs to be available.

In some seminar games, the MMT may simply be the best judgement of the facilitator at the point an adjudication is required. When a facilitator adjudicates some activity, the rationale for this should be part of the records collected for analysis.

Scenario Components

As issues, sub-issues, and EEAs are compiled in a DCMP, practically each row in the spreadsheet should call for items to be added to the scenario. For example, if one of the sponsor's issues is over the effectiveness of close air support for troops on the ground, then there should be a component of the scenario that has close air support missions.

See below for a mechanism to check that the DCMP and scenario are consistent with each other (the MSEL Crosswalk).

It is best that scenario writers and analysts collaborate as the DCMP and the framework of the scenario are developed. Scenario writers should be provided with access to the DCMP as the decomposition of the sponsor's objectives and issues will provide scenario writers with the rationale for including events and injects to satisfy the analysts' requirements. Similarly analysts can check off their EEAs as corresponding elements of the scenario develop.

One area of vulnerability of a game design is if the analysts and the scenario writers do not agree on game objectives and issues. Sharing the content of the DCMP is an excellent way to confirm there is agreement on objectives and issues -- or to expose disagreements early, if there are any. The scenario writers do not need to adopt the DCMP for their own use if they have something equivalent. But often they do not and it can give them considerable support as they develop a framework for the scenario and then start to fill in details.

Methods, Models, and Tools

A column within the DCMP spreadsheet can be set aside for the Methods, Models, and Tools (MMT) that pertain to a particular issue, sub-issues, or essential element of analysis. If the game is a short small seminar game the only MMT used throughout could be "Facilitator's Judgment".

But more elaborate MMT can also be used. For example, if a sub-issue represents some aspect of ammunition consumption, the associated MMT might be a combat model that can represent engagements and track rounds fired. In another example, where the medical treatment of personnel casualties may be the focus, analysts may have a simulation that incorporates the extraction of casualties from the battlefield, their transport to medical facilities, the treage to determine how they will be treated, and then how they are handled by the medical staff.

Once the column associated with MMT has been completed, it can be used to manage the inventory of MMT. For example, if a combat model is a requirement within the DCMP, then the analysts and designers will have to allocate resources to ensure it will work as intended within the game structure.

Note that even if aggregated MMT amounts to a repeated use of "Facilitator's Judgment", the facilitator and the team must ensure that the facilitator has appropriate competency to support the full range of issues that may emerge during play.

Scheduling Events

The DCMP provides a mechanism to schedule certain events. There can be "Location" and "Time" indicators in the right hand columns to indicate where, during a scenario or in a MSEL (master scenario event list), an opportunity would arise to observe the issue in question. If the issue were important and there had been no opportunity to collection reactions to it, then the MSEL would have to be adjusted accordingly. Hence, the scenario design must accommodate the analysis plan.

For smaller games, the use of the DCMP for scheduling seem be of little value. However it can provide a checklist to ensure that issues that are expected at a particular point, arise at that point. If they do not, the facilitator or control cell will have to face a decision to introduce the issue at some other time, or to accept that no evidence on that issue will be captured.

For larger games, say where there are many players and they are dispersed in a training area, or where the play extends over several days, the DCMP scheduling mechanism assists in the economical management of data collectors and observers: they will be deployed at the right time and right location.

Various Extensions to the DCMP

While not necessary within a DCMP, a developing DCMP may be elaborated with various additions. For example columns can be added for the following:

• titles of doctrine or training manuals that should be consulted so observers have more contextual information on some potential event

• external agencies that might need to be consulted or that might need to approve some activity, e.g., range control or air traffic control.

DCMP-Scenario Crosswalk

MSEL Crosswalk format The Analysis Handbook also provides a mechanism to crosswalk from the issues that are the focus of a war game to scenario design -- from the Essential Areas of Analysis within the Data Collection and Management Plan (DCMP) to the Master Scenario Events List (MSEL). This is the means to do a double check that important issues for data collection will be afforded opportunities within the scenario to see how participants deal with them. Note that it is probably risky to have an important issue without some corresponding activity in the MSEL, although it could come up merely by chance. However there may be entries in the MSEL that have no corresponding issues from the DCMP; specifically, there may be items in the MSEL that provide some extra realism for the players, or that can provide some other secondary function, e.g., events to develop team cohesion or to practice a routine that may be necessary later in the scenario.

Copyright: MMXVII, Fred Cameron,