Data management: Data collection, data analysis and reporting
Data managment runs almost throughout the whole exercise cycle, from preparation, to conduct, analysis and reporting, even if evaluators are not heavily involved during the planning phase. It is useful to provide some guidance to evaluators for the content of data collection, usually this is developed in two pillars: evaluation of players performance and exercise conduct. There are different methods for data collection: through evaluators input, feedback from participants, recording, possibly input from observers, interviews, debriefs. To decide if and how to use observers for data collection, exercise managers may decide differently based on who the observers are. Info from observers may support evaluators, depending on the design of the exercise.
Mainly raw data, not information, that is available.
Potential Solutions
- need of involvement of evaluators in exercise plannings as soon as possible (chief evaluator)
- training on how data management is performed before the exercise is performed
- provide an exercise evaluation tool to support evaluators by providing the necessary data, the timeline of the exercise, contact points to other evaluators, information on the scenario
- MODEX: not only observers, but also trainers - seperate data sets and data source, to be considered
- chief evaluator - involve in planning? - input from evaluation after exercise
Rationale & related CM function(s)
- Pre-defined checklists or forms to log observations throughout an exercise
- Possible options of documenting perceptions (e.g. text, photos, videos, audio recordings, drawings, map annotations, etc.), which can be utilized for evaluation
- Tools and methods to process perceptions in order to generate evaluation insights (e.g. through efficient compilation, analysis, data visualization, etc.)
- Events (e.g. Workshops) to discuss and analyse findings
- Chosen exercise type, such as Tabletop Exercise (TTX), Command Post Exercise (CPX), or Full Scale Exercise (FSX)
- Defined Roles and Functions within the exercise organisation and evaluation efforts
 |
Portfolio of Solutions web site has been initially developed in the scope of DRIVER+ project. Today, the service is managed by AIT Austrian Institute of Technology GmbH., for the benefit of the European Crisis Management. PoS is endorsed and supported by the Disaster Competence Network Austria (DCNA) as well as by the STAMINA and TeamAware H2020 projects. |