Wednesday, October 3, 2018

Bureaucratic Politics and Decision Making Under Uncertainty in a National Security Crisis: Assessing the Effects of International Relations Theory and the Learning Impact of Role-Playing Simulation at the U.S. Naval Academy

Author: Nikolaos Biziouras
Date: 2013
Summary & Critique by: Bryant Kimball

In this journal article, United States Naval academy professor Nikolaos Biziouras measures the learning impact of role-playing in an international relations course using a pre/post-test research design.  Before diving into the exercise, Biziouras outlines the previous literature in favor of role-playing and active learning, highlighting the following outcomes from the previous research.  Students who are enrolled in courses with an active learning component have:
  • Acquired a deeper, longer-lasting knowledge 
  • Become more enthusiastic about the material
  • Enjoyed the application of theoretical knowledge to empirical circumstances
  • Increased in skill development
The literature also concludes that skill development includes:
  •  An increased capacity for public speaking   
  • Appreciation for others’ viewpoints
  • An expanded ability to conduct independent research 
  • A unique opportunity to shift from a passive recipient of knowledge into an active participant
  • A heightened realization about the importance of critical thinking, especially with unexpected challenges and unfamiliar environments
Yet, Biziouras makes clear that active learning exercises don’t come without their fair share of limitations and requisites. Increased use of role-playing emphasizes the importance of constructing a methodologically sound exercise, including clear rules of engagement, and well specified and realistic learning objectives.   Also, the facilitator must be aware that a break from the traditional routine may be outside of some participants’ comfort zones. Furthermore, an unstructured exercise may increase “social loafing.” What Biziouras finds most important is some form of debriefing so that both participants and facilitator can evaluate the experience based on whether or not the learning objectives were met.  

Understanding these requisites, Biziouras constructed his own role-playing simulation for his class at the U.S. Naval Academy, and tested the effectiveness of the simulation using a pre/post-test research design.  Some of the questions in the pre/post-tests include:

·         “How do you think leaders should deal with foreign policy crisis in international relations?”

·         “What do you think are the most common mistakes that leaders make in a foreign policy crisis?”

·         “Do you think that changes in leaders lead to changes in outcome in the international system?”

Biziouras was able to measure the effect of role-playing on his students based off the differences in pre and post-test responses.  


·         Biziouras’ article emphasized the advantages of role-playing in applying theory.  As a current graduate student in an applied intelligence program, I can appreciate the value in that.  Judging by his brief review of previous literature of the effectiveness of role-playing, this methodology appears to be one of, if not the most tested methodology our 2018 AAT class has researched so far.  Although without diving into the methodology of the literature Biziouras mentioned, I found the advantages, the list of skills developed, and the limitations and requisites to be helpful in evaluating role-playing.  Biziouras’ own test of the methodology was sound, however he did not factor in the changes in student response between pre and post-test resulting from his students’ expectation to changing their answers from pre to post test. I emphasized a specific skill that I found to be of utmost importance to intelligence analysts: A heightened realization about the importance of critical thinking, especially with unexpected challenges and unfamiliar environments.  As analysts are always attempting to uncover information about deceptive, challenging, messy, and unfamiliar environments, role-playing may be an effective methodology to employ.


  1. Adding to your highlighted point-- I think role-playing alerts individuals to their ability to make sound judgments in those unexpected challenges and unfamiliar environments. There are times when you can reference personal experience or existing expertise regarding a challenge or environment; there are times when you have little to no frame of reference. That doesn't mean you can't make decisions. A theatre director might say, "it's called acting for a reason!" Role-playing can increase an analyst's confidence in making judgments amidst uncertainty.

  2. Role-playing is definitely an effective method when it comes to training for an operation or else it would not be employed so much. It seems crucial that a simulation must be well-organized in order for it to be as beneficial as expected to be. Making the exercise as realistic as possible will put the situation into perspective for trainees and make it more valuable for a successful operation.

  3. Bryant,
    I agree with your decision to emphasize critical thinking in unfamiliar environments. In particular, the indirect benefit of conducting role-playing is that increase in confidence which is transferable to all situations. In this program that trains analysts as "generalists", do you think role-playing should be used more often out of our toolkit?

    1. Tom - I think there are plenty of elements of role-playing that feed into this program's "generalist" design. But I would argue the generalist philosophy centers on growing a toolkit with a range of methodologies for the analyst.

  4. I struggle to see the practicality of role playing as an intelligence methodology. It seems like the research simply states it is a good teaching method for understanding theories and people like it because its active. The author states "A heightened realization about the importance of critical thinking, especially with unexpected challenges and unfamiliar environments." Don't you think "importance of crtical thinking" is kind of a no duh in the field of intelligence?