Modeling for Decision Making Under Uncertainty In Energy and US Foreign Policy

Pages 136-148
Author: Lauren C Culver
August 2017
Critique: Bryant C Kimball

Summary

Standard Monte Carlo simulations consist of a collection of hundreds of thousands of random functions that cycle through a given distribution.  The results of these thousands of interactions is considered the generation of potential outcomes.  While historical risk assessment forces the analyst to consider possible outcomes based off of all the possibilities that have already happened, the Monte Carlo method combines stochastics and simulation; cycling a random sampling of inputs into a virtual representation of a problem over and over and over again to obtain a distribution of results. 

In 2017, analysts supporting decision making at the intersection of energy and U.S. foreign policy teamed Monte Carlo analysis with decision analysis, predictive scenario analysis, and exploratory modeling to understand the threat of sixteen countries’ energy demands on US foreign policy.  However, the experiment itself pits the four models of uncertainty analysis against one other with the intentions of issuing a recommendation about the most appropriate approach to uncertainty analysis for foreign policy. 

Figure 1

The model run time for a single country took about 20 minutes to produce 1,000 unique input combinations for each policy (Figure 1)), so 6 hours for all sixteen countries, not including time for analysis.  Although run time is lengthy, when the results are calculated there is both a clear sense of the best policy, the one that maximizes expected net benefits, and the risk associated with it and there are hundreds of scenarios to compare. In addition to providing the analyst with rich results, the output of Monte Carlo analysis is intuitive by nature and the visuals make it easy to detect the trends that lead to analysis (Figures 2 and 3). 

Figure 2

Figure 3

The simulation produced a series of outcomes for each country as well as policies to match (Figure 4). The researcher found that the results of the Monte Carlo analysis correctly convey the policy and that one policy will not always be beneficial. It requires additional, time-consuming analysis by the analyst to more specifically identify the input set that drives net benefits of a particular policy.

 

Figure 4

Critique

Essentially, Monte Carlo simulations can present an analyst with a distribution of outcomes for a given situation.  This research serves as evidence that Monte Carlo simulations can help reduce uncertainty as well as issue recommendations based on specific outcomes across a distribution.  This validated tool clearly reduces uncertainty by allowing the analyst to easily deconstruct the potential outcomes. Even more so in intelligence analysis that forces the analyst to discriminate between numeric ranges of likelihood, Monte Carlo as a tool, can help add to an analyst’s judgment by either validating or helping guide the analyst to the most accurate WEP. 

What this particular model does an even better job of depicting is that no one methodology is holistic.  Each uncertainty model relies on the others to complete the picture. This holds true with all the methodologies we’ve studied thus far, and highlights the importance of building a toolkit with a wide range of methods and modifiers to use when deemed relevant. 

https://ngi.stanford.edu/sites/default/files/Culver_dis%5B1%5D.pdf

Summary of Findings: Social Network Analysis (3.5 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in November 2018 regarding Social Network Analysis as an Analytic Technique, specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Social Network Analysis (SNA) is a method for analyzing social structures based on connections/relationships (links) between individuals (nodes). It is applicable to a wide variety of disciplines, demonstrating its flexibility. With a robust data set, the visual outcome can be exceptionally detailed. Regarding the data-- SNA requires structured data, and expertise in manipulating that data accurately. While a moderate level of knowledge and expertise is necessary to properly execute this method, getting there is fairly simple.

Strengths:

1. Provides a holistic picture of a network
2. Maps the spread of ideas
3. Determines who the most influential individual(s) in a network is
4. Provides an alternative view where attributes of individuals are less important than their relationship ties
5. Can help analysts identify the most powerful nodes in a network
6. Can be used to simulate how information ripples through a network
7. Can be used to identify intelligence gaps

Weaknesses:  

1. Defining relationships and understanding what that relationship means is difficult
2. Visualization does not always tell a clear story, making it difficult to communicate to a DM
3. SNA for ambiguous organizations may cause confirmation bias
4. Constructing a matrix can be time consuming
5. Matrix attributes can be subjective and often biased
6. This tool can only be used in some situations, not all cases are applicable for social network analysis

How-To:

Exploring the full scope of social network analysis and how to do it properly was beyond the scope of this assignment.  Therefore, a simplified form of the process was developed that would allow participants in the exercise to experience some of the challenges as well as the benefits of performing a social network analysis.  The “rules” of this simplified process are below:

1. Write out the names of the last 10 people you exchanged text messages with
1. Exclude group chats

2. Pick one person to write the names of those 10 people around his/her own name on a sheet of paper or whiteboard, with ample space to draw connections between names.
1. The first person is to then connect his/her own name to all those in his/her own network.
2. The first person will then connect those in his/her network to each other based on whether they know each other.  

3. The second and third participants draw out their own networks repeating steps 1 and 2
1. Important: any repeat names within each other’s network can pose a problem. We recommend omitting repeat names if a person is already listed in another participant’s network. The connections will ultimately be made in step 4.

4. Connect participants’ networks to each others based on whether they know each other.
5. With all people listed in the network, build a matrix of network connections.
6. Build a frequency table to determine who in the network has the most connections.

The participant with the most connections is the most involved in the social network.

Application of Technique:

The Advanced Analytic Techniques (AAT) class was presented the following scenario to conduct a social network analysis: A transfer student was new to the program and on the first day sat at a table with 3 classmates. The task for the activity was to determine which of the classmates at the table had the most connections to others in the class’s network. For the instructions the AAT class was given to complete the task, see the “How-To” section.

Figure 1. Individual Networks

Figure 2. Connecting individual networks to each other

Figure 3. Matrix based on relationships

Figure 4. Frequency Table

For Further Information:

1.     Social Network Analysis: ‘How to guide’

2.    Social Network Analysis Activity and Explanation

3.    Social Network Analysis: A Methodological Introduction

Social Network Analysis in the Study of Terrorism and Insurgency: From Organization to Politics
Steven T. Zech and Michael Gabbay
Summary and critique by Jillian J

Zech and Gabbay conclude that social network analysis (SNA) has made significant contributions to understanding militant operations, particularly regarding the merits and implications of "centralized and decentralized structures, the relationship between efficiency and security, the network signatures of key individuals, and the factors that shape network structure." They go on to criticize the lack of alignment between theoretical and empirical studies about SNA, calling for better integration of network concepts and more precise data and a focus on temporal factors.

Their study "applies network analysis to intrinsically political questions, entailing a focus on group-level nodes engaged in the same conflict." They focus on militant fragmentation, specifically infighting, outbidding, alliance formation, and group constituencies. They discuss how the similarities in terminology highlight a hybrid nature of fragmented militant groups wherein aspects of international systems blend with political party competition e.g. rival constituents or winning a "state".

The authors aim to enhance the understanding of fragmented civil wars and insurgencies. The charts below shows their connections and findings from other papers and cases.

Critique
Zech and Gabbay's analysis was insightful and organized. I think they did a good job of pulling specific analytical claims from each case to support what could have easily been a long-winded analogy. I do wonder if it's a situation similar to the saying, "when you have a hammer, everything looks like a nail." Even so, the points they assert through the organized charts are useful in understanding SNA in terrorism and insurgency cases. 

The Social Network of Hackers

Authors: David Décary-Hétu, Benoit Dupont

Institution: University of Montreal

Journal: Global Crime

Date: August 2012

Summary:

The premise of the article is to assess the value of social network analysis in criminal investigations especially as it has to do with hackers and cybercriminals. Canadian law enforcement provided the authors raw data from the criminal investigation of a group of hackers operating botnets for commercial gain. The authors used the raw conversation logs from the arrested hackers’ computers to conduct the analysis.

The authors begin the article by explaining the value of social network analysis in criminal investigations. The authors state that in comparison to criminal organizations of the 1970’s, today’s criminal enterprises bear little resemblance in form and function. Modern organizations are more resilient and robust whereby removing important members from the network doesn’t necessarily destabilize the rest of the network. Since important members may not destabilize operations, understanding the entirety of the network becomes a relevant and necessary task.

In the modern age of information gathering, intelligence and law enforcement have an increased number of tools they can use to generate and process information about networks. The authors believe that social network analysis is suitable to process large sets of structured and unstructured data to better understand opaque networks seen commonly in criminal organizations but also potentially in more unformal organizations like those seen online.

The data provided by Canadian police included 4714 messages between individuals in the hacking network under investigation. The authors identified 771 individuals including the 10 arrested hackers in the network. The authors decided to focus the analysis on 38 individuals: the 10 arrested hackers and 28 other Persons of Interest.

The authors examined conversations between the hackers and the POI’s to determine the centrality and power of the individuals in the network. The authors assess centrality through looking at ingoing and outgoing messages between individuals in the network. The pattern that is revealed is an indicator of status or prestige in the network. Additionally, the authors assessed centrality using “flow betweenness centrality” which posits that the more often an individual is located between other individuals, the more important they are in the control and flow of information within the network.

The authors recognized some significant limitations in their study, specifically that they may not have a full record of conversations between individuals in the network. Additionally, the conversation records were only between hackers or hackers and POI’s. The authors did not have conversation records between POI’s.

Figure 1. The social network of the arrested hackers, shown in red, and Persons of Interest (POI's), shown in blue. 

The authors built a social network using the data, which is shown above.

The authors state:

“In Figure 3, the squares represent the arrested hackers and the circles represent POI. A few nodes stand out in this graphic. N505 (bottom right) looks isolated from the network and even more so from the other arrested hackers. Although some arrested hackers seem to have many contacts (N2 and N29), others have very few ties (N73 and N505). Moreover, some POI – such as N217 – seem to have a more central position in the network than some of the arrested hackers. This suggests that some important players might have been ignored while some ‘fringe’ people were arrested.”

Initial conclusions derived from the pattern of conversations suggested that the network was broken into groups of highly connected, mildly connected, and disconnected hackers. Overall the network was not very cohesive or connected, shown by an overall low number of messages between 2004 and 2008. Based on their analysis of the pattern of conversations, the authors believe the arrested hackers were less involved in the group based on their connectedness.

The arrested hackers were also more connected with POI’s than other hackers, which the authors believe is indicate of either the network was tightly organized or whether part of the network was overlooked. The premise of the second hypothesis is that some of the POI’s may be more important to the network than the hackers that were arrested.

The authors ranked individuals in the network using degree centrality coefficients. Of the top 10 individuals, 8 were arrested. The other 2 ranked poorly. Using flow betweenness centrality, which helps to identify information brokers in the network, the authors identified that the top 7 individuals were among the arrested hackers, suggesting that the police targeted individuals who control the information in the network.

In contrast the authors found that while the hackers acted as brokers of information, they were not necessarily the most powerful actors in the network. POI’s were assessed to have more power in the network. The authors state “this indicates…[hackers] are not positioned so efficiently in the network as to be indispensable. The actors they are tied to usually have other alternatives to get information they need…” The authors conclude this section by suggesting that the social network of these actors resembles an association between individuals rather than an organization.

By targeting the information brokers, the arrested hackers, the police were able to disrupt the network, was borne out by the social network analysis. Being brokers meant that the hackers were more visible in the network, increasing their chance of arrest. While the authors found that these individuals were most central, they were not the most powerful. Again, the POI’s were shown to be more powerful within the network. Even though the POI’s were assumed to be periphery in the network and overlooked, the analysis revealed that a select number of POI’s may have been more important in the network because of their power rather than their centrality or visibility.
The authors conclude the piece by suggesting that the police need to have an intimate knowledge of how hacking communities function to determine how is relevant in the network beyond their visibility. The authors further recommend that the police should monitor the powerful individuals in the network, the POI’s, to detect how the network rebuilds after the arrest.

Critique:
Given the data available, the authors were able to identify individuals in the network that may have been of more importance than the hackers that were arrested. I believe that this highlights an advantage to social network analysis when investigating opaque networks. Given that the most visible or connected individuals in the network may not be the most powerful, the police may have overlooked key individuals based on their intuitive judgement of the evidence. The authors state that after the arrest of 6 of the 10 hackers, the fragmentation increased exponentially. After the 6^th arrest, subsequent arrests had much more limited effects on the network fragmentation. This could have been the result of misidentifying key players in the network. By suggesting this, the authors indicate that one of the neglected POI’s may have been more worthwhile in investigating further prior to making the arrests in order to destabilize the network.

At the same time the authors highlight the key problem and limitation which is the availability and abundance of the data. The authors had the cooperation of the investigation authorities who provided the data, which enhances the efficacy of the study. At the same time, the authors believe that if they had more information that would have resulted from a deeper investigation of  the POI’s, they might have been able to draw additional conclusions as to the importance and relevance of the POI’s.
Overall, social network analysis was able to gather impressive insights using the limited data, showing its relevance in understanding networks with a limited amount of information.

Link 1: https://www.researchgate.net/publication/241711312_The_social_network_of_hackers
Link 2: https://www.tandfonline.com/doi/abs/10.1080/17440572.2012.702523

Social Network Analysis of an Urban Street Gang Using Police Intelligence Data

Social Network Analysis of an Urban Street Gang Using Police Intelligence Data

Authors: Daniel Gunnell, Joseph Hillier, Laura Blakeborough

Summary & Critique by: Billy

The authors of this paper employed social network analysis as a way of better understanding how gangs operate and assist police in allocating their resources.  The researchers developed two research 

questions (Gunnell et al., 2016):

1   1.. What can social network analysis tell us about gangs?

     2. How useful are the social network analysis outputs for the police?

The network began with 5 individuals who were known by police to be gang affiliated.  The researchers followed the following steps to construct their social network (Gunnell et al., 2016):

1             1. Researchers collected police intelligence data on the 5 gang-affiliated individuals dating back 6 months. These individuals were the “primary individuals” for this research.

2            2. Any secondary individuals mentioned within the data were identified

3            3.  Relationships between primary and secondary individuals were coded

4            4. Researchers collected police intelligence data on secondary individuals dating back 6 months.             This data was then coded.

5           5. Other individuals that were mentioned in data were identified

6           6. Relationships between the “other” individuals and secondary individuals were coded

The relationships were coded based on a framework that recorded charged or suspected criminal relationships, non-explicit criminal relationships and the nature of the relationship (Gunnell et al., 2016).

According to the authors, the analysis produced a network of 137 individual nodes that included a previously unknown link between two gangs.  Although not all of the individuals within the social network analysis are gang members/criminals, it provides police with knowledge of those who may be at future risk of gang activity. 

The individuals circled in red are the original five members who are known gang members (Gunnell et al., 2016).

Critique:

I agree with the authors when they state that social network analysis can provide a solid framework of understanding a groups structure when little is previously known.  They also state that it can remove bias in constructing a network because little experience related to the targeted group is required to populate the data.  From the framework of anonymous nodes constructed by the authors, it is easy to see how analysts can build a very detailed network by including photographs, personal data, etc.

 Gunnell, D. Hillier, J., Blakenborough, L. (2016). Social Network Analysis of an Urban Street Gang Using Police Intelligence Data. Research Report 89. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/491578/horr89.pdf

Summary of Findings: Speed Reading (3 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in October 2018 regarding Speed Reading as an Analytic Technique, specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Speed reading is “a set of  active, mindful and conscious strategies that allow a person to speed up what they are reading”. Methods for improving/teaching speed reading include skimming, scanning, satisficing, the Evelyn Wood Method, RSVP, track and pace method, pointer method, etc., all of which are designed to help you read faster.    

Strengths:

1. Increases the speed at which you read (e.g., number of words per minute).
2. Helps the reader to identify important parts of the text more quickly.

Weaknesses:  

1. Reading comprehension is often adversely affected by speed reading.  
2. Readers are likely to skip over details that may affect the analysis of the text.
3. Effectiveness of speed reading is contingent upon the difficulty of the material.

1. For example, scientific journal articles, government publications, and online news articles all vary in reading-level difficulty because they are written with different styles and for different audiences.  

How-To:

There are a number of speed reading techniques that have been proven to increase reading rates to some degree.  The three techniques employed in class included:

1. An eye-scanning training exercise (Youtube),
2. Placing a notecard above the line one is currently reading forcing the reader to continue to move eyes down the page, and finally,
3. A meta-guiding technique where the reader traced the current word using a pen in order to guide the eye to move faster along the text, avoiding fixating on a single word or phrase.

Application of Technique:

The class conducted an exercise to test two speed reading methods. Students first assessed how many words were in the first five lines of our reading material. As an initial exercise, students read their reading material for 1 minute, marking start and ending points.

Following the initial exercise, students were shown a Youtube video to warm-up and prepare their eyes for using our chosen techniques. The video showed an object, in this case a pumpkin, move to different points on the screen over the course of a 1.5 minute period. Students were required to follow the object with their eyes, without moving their heads as the object moved.  

The first technique applied was the Card method. Students took an index card and placed it above the words/sentences being read. As students completed each sentence, they would move the card down the page.

The second technique applied was the Pointer method. Using this method, students would take a pen or pencil and place the tip  under the line being read. The aim of the technique is for the student to follow the tip of the pen/pencil as they read, encouraging them to move along the line, left to right.

In each method, students were advised to try and increase reading speed over the course of the allotted 1 minute time frame for the exercises. At the end of each exercise, they noted how many lines they were able to read.  Additionally, students were advised to focus on their speed, as opposed to the comprehension of the material being read.

At the conclusion of the exercises, students assessed how many lines and words they read during the initial exercise reading normally, reading using the Card method, and then reading using the Pointer method. On average, students read faster with the Card and Pointer methods in compared to the initial exercise where they read normally.   

For Further Information:

The effects of reading speed and reading patterns on understanding of text read from screens  

Speedreader11 Youtube channel

So Much to Read, So Little Time

Speed Reading: You Can’t Always Get What You Want, but Can You Sometimes Get What You Need?