Friday, September 5, 2014

The Evaluation of Role Playing in the Context of Teaching Climate Change


This article presents a study done by Belova, N., Feierabend, T., and Eilks, I. to evaluate role playing in science-based scenarios. Belova et al. stipulates that although many applications for role playing in science education have been developed, there is limited research on the performance during the exercises. Additionally, Belova et al. points out that there is no agreed upon method for analyzing arguments within a role playing exercise. This paper presents a systematic method for analyzing performances in role plays and what level of argumentation and decision-making capabilities are evident within the exercises.

Belova et al. performed the study by having 20 groups of students, aged 15 to 17, participate in the role playing exercise of debating climate change as a stake holder, such as the automobile industry or Greenpeace activists. The 20 groups were assigned to one of four class subjects, biology, physics, chemistry, or politics which then became their primary source of knowledge for the exercise. The exercises were recorded and evaluated by two analysts.

The arguments made by students during the exercise were evaluated in three ways: domain, level, and reference. Domain pertains to where the student derived the argument from, level refers to the complexity of the argument, and reference is the ability to which a student can refer back to comments made by previous speakers. A high score was awarded for arguments sourced by scientific facts, if they were interconnected with multiple justifications, and if it was made within the context of the conversation at hand. A low score was assigned to arguments with no clear origin and unrelated to climate change.
Of the 20 role playing exercises, only 20 percent were classified as high quality and reflective. Additionally, only 20 percent of the arguments made during the course of the exercise were supported with scientific facts. Furthermore, the scientific based arguments possessed little variety, used terminology incorrectly, and were incorrect 10 percent of the time. Although the quality of the arguments was lacking, the analysts found that 80 percent of the conversations flowed smoothly.


The use of role playing did facilitate the exchange of ideas among the students; however, it did not improve the quality of the conversation significantly. The students, rather than citing scientific facts which were provided to them, resorted to citing socio-life experiences to bolster their claims. This is likely to do poor preparation and lack of practice rather than a true belief that one’s personal experiences carry more weight.
Additionally, age among the study participants is a factor that must considered. The study cites that the older students produced better arguments than the younger groups. It is likely the older students had more knowledge from school to draw from in their arguments, whereas the younger students became dependent on the limited amount of info provided to them.
Finally, a factor that likely impacted the study was the role of the moderator in each of the exercises. A different moderator was present for each exercise, and that likely influenced the results of the exercises depending on style of the moderator.
Belova, N., Feierabend, T., and Eilks, I. 2013. The Evaluation of Role Playing of Teaching Climate Change. International Journal of Science and Mathematics Education.


  1. What sort of scale did the rubric for evaluating level contain? If level speaks to the complexity of the argument I am interested in how the authors measured complexity. Does the article state if domain, level, and reference are weighed equally in assigning a score?

    1. Level was assessed on a 6-step scale:
      L0: Not Related
      L1: One relevant argument was provided
      L2: Two arguments were provided without relation
      L3: One relevant argument with justification
      L4: Two or more arguments provided in a logical chain, justified by facts
      L5: Provide arguments with justifications and draw conclusions from arguments interconnectedness

    2. Additionally, the analysts scored reference, domain, and level independently of each. Therefore, a score would look like this: Rx:Dx:Lx.

  2. It was interesting to see the authors using students aged 15 to 17 to participate in the role playing exercise of debating climate change since many of them should have had knowledge in the subject matter-which is one of Armstrong's rules for role-playing. Since the students were assigned to one of four class subjects, biology, physics, chemistry, or politics the results seems to be lower than expected. Although the students were not experts, the results were definitely lacking. I agree with you that the older students could have produced better results due to their increased knowledge and material of the subject matter.