Saturday, October 28, 2017

BAYESIAN INTELLIGENCE ANALYSIS

Dr. Barbieri Davide of the University of Ferrara examined the Bayesian approach to intelligence analysis by first looking the the three ways of examining probability. He showed the historic approach from French mathematician Laplace with the equation P(E)=m/n. The probability as shown is the number of favorable cases divided by the number of possible cases. In this approach, he notes that the probability of an event will always be between 0 and 1. The second approach he examined was the frequentist approach, also known as the "law of large numbers. This equation divides the frequency of events by the frequency of events out of "n" trials. However, analysts do not use this approach as much because they often work with less data than what the frequentist approach requires. The third approach is the amount of confidence an analyst gives an estimate based on previous experience and available information. He also goes into conditional probability and how something is likely to happen if something else happens.

After examining  regular probability, he examines the Bayes approach to probability. He shows the conditional probability equation first to provide a comparison. This equation is P(H/E)=P(HupsidedownUE)/P(E). The Bayes approach to probability is P(H/E)=P(E/H)P(H)/P(E). P(H/E) is revised probability after reviewing evidence. P(E/H) is the conditional probability of E in case of H. P(H) is the probability without any evidence. The Bayes approach allows for the estimate to change as more information comes in.

In intelligence analysis, the equation given is R=PL with R being the estimate of conditional probability of the hypothesis H after revising evidence E. P or prior estimate times the likelihood of an event or L. He advocates that this approach is best used in strategic warning; for example the probability of a terrorist attack. This approach forces analysts to quantify their estimates and reduce cognitive bias using competing hypotheses. The weaknesses of this approach in the vulnerability to false evidence, time constraints, and limited information.

He uses historical events such as the Cuban missile crisis and the tension between Russia and China during the Cold War to show the differences between conventional probability and the Bayes approach. The estimates were the same, but the analysts using the Bayes approach arrived at their conclusions faster than the conventional analysts.

Critique: Dr. Davide was very thorough in his study of the use of Bayesian networks in intelligence analysis. However, it was a difficult read because he uses mathematical language very often. His examples were also slightly outdated. An updated version of his study would be useful in the modern intelligence field.

Dr. Barbieri Davide. https://www.researchgate.net/publication/257933578_Bayesian_Intelligence_Analysis.

 

FREQUENTIST AND BAYESIAN STATISTICS: A CRITIQUE

By
D.R. COX
Nuffield College, Oxford OX1 1NF, UK

Summary:

The paper discusses the distinction between frequentist and Bayesian approach for statistical inference where they consider historical background discussing the evolution of the approaches over time. The paper continues to discuss the critique on frequentist approach and discusses the two contrasting Bayesian views. The difficulties with the notion of a flat or uninformative prior distribution are discussed.

Critique of Frequentist approach

Advantage:
  1. It provides a methodical approach to a wide range of statistical methods and does not require additional specification beyond that of the probabilistic representation of the data-generating process.
  2. It provides a way of assessing methods that may have been suggested on relatively informal grounds.
Disadvantage:
  1. The problem in principle in frequentist formulations is that of ensuring that the long-run used in calibration is relevant to the analysis of the specific data being analyzed. Proposed solutions are applicable in only certain limited situations. 
Critique of Bayesian Methods

The paper discusses Bayesian methods we have to extend the notion of probability so that we can specify a prior distribution for the unknown constant. There are two radically different ways of doing this.
  1. Personalistic theory: This approach has the ambitious aim of introducing into the quantitative discussion uncertain information of a more general kind that is represented by statistical data in the narrow sense. There is an emphasis on trying to achieve self-consistency and coherency in probability assessments.
  2. Probability as Rational Degree of Belief: This approach involves a notion of rational degree of belief in an attempt to address the question of assessing the evidence in a specific dataset by seemingly being indifferent or representing ignorance to focus attention on the data.
Some of the difficulties with Bayesian Statistics are:

• Finding the prior weight is often complicated
• The nuisance parameters must be arranged in the sequence of importance, even though none of them is of intrinsic interest
• If the parameter of interest changes the whole prior structure may change
• If the sampling rule or design changes the prior will in general change
• It is emphasized that the prior weights are not to be thought of as prior probabilities, raising a question-mark over the interpretation of the posterior
• Many of the formal simplifications arising from all calculations being probabilistic are lost.

Critique:

The article provides various situations in which both approaches may not be ideal when applied statistically depending on the type and/or complexity of the data. Both approaches clearly cannot be treated as one-size fits all when applied to various kinds of datasets. The statistician bears the burden of understanding the most suitable approach that will yield the most appropriate results. As more extensive studies are conducted, these approaches will continue to evolve, as new ones are developed.


Friday, October 27, 2017

Bayesian statistics: principles and benefits

Matt Haines

Summary:

     In this article Anthony O'Hagan analyzes the differences between Bayesian statistics and traditional frequency statistics. He does this by defining the theory of Bayesian statistics and then outlining its strengths and critiques. O'Hagan begins by defining Bayesian Statistcs, saying that Bayesian starts by creating a statistical model to link data to parameters. This Is the only step where Bayesian and frequency statistics are similar. Then Bayes formulates prior information about the parameters. Next it combines the two sources of information and finally it use the resulting posterior distribution to derive inferences about parameters. This process leads " to less pessimism when the data are unexpectedly bad and less optimism when they are unexpectedly good".
     The author then begins to critique Bayesian statistics. O'Hagan demonstrates that the reason Bayes is so controversial is that it is inherently subjective. By using prior information, Bayes becomes subjective because one researchers prior information may be different than someone else's and therefore replicability becomes a problem. O'Hagan then states that Bayes accounts for this by scaling and assessing the prior data for subjectivity. He also states that frequency statistics are hypocritical in there assumption that frequency statistics assume that they have. O'Hagan also states that frequency statistics are usually misinterpreted.
     At the end of the article O'Hagan outlines the total benefits of the Bayesian approach to statistics. Which are that Bayes gives a more direct, intuitive and meaningful statement of the probability that the hypothesis is true, that Bayesian methods can answer complex questions cleanly and exactly, that no relevant information is omitted in a Bayesian analysis, and that Bayes can quantify uncertainties for decision makers.

Critique:

     This article really does a great job of explaining how Bayesian Statistics are done and the theory behind the method. It gives a number of resources for learning about the application of Bayes and some critiques and analysis of frequency statistics that are rarely brought up. However, I did not think that this article truly critiques Bayesian statistics. The author ever really accepts the weakness of Bayes that there is no real way to manipulate the prior information to make it uniform, and that in order to perform a good Bayesian analysis you need to have serious computational power. It seems to me that Bayes can be a great tool but until more research is done to refine it it will only ever be just another regression method to use.

http://library.wur.nl/ojs/index.php/frontis/article/view/856/422

Thursday, October 26, 2017

Bayes' Theorem for Intelligence Analysis

Summary and Critique by: Jared Leets

Summary:
The author, who was an intelligence analyst for the CIA, begins by explaining why intelligence analysts should be interested in probability theory. Intelligence analysis should typically must be undertaken on with insufficient evidence. Bayes' Theorem in its form served participants in the intelligence analyst’s test program as their distinguishing rule for evaluating new evidence and material. In the article the equation R=PL as the odds-likelihood formulation of Bayes' Theorem was used. The author stated that R is the revised estimate of the odds favoring one hypothesis over another, the odds after contemplation of the last piece of evidence. P is the prior estimate of the odds, the odds before contemplation of the last piece of evidence.  Once the estimate was ready, the analysts participating did not make any judgments regarding P. The participating analysts gave only insight about about L, which happened to be the likelihood ratio. The author stated that the likelihood ratio was the analyst’s assessment of the distinguishing piece of evidence.

The author stated that there exists three features of Bayesian probability that differentiate it from conventional intelligence analysis. The first is that the intelligence analyst is asked to quantify judgments which he or she does not typically do in numerical terms. This feature is what draws the most criticism against Bayesian in the intelligence community. Analysts must disagree in their opinions of the exact figure that shows the distinguishing value of a piece of evidence. A proponent of Bayesian might say that disagreement among analysts is simply a characteristic of traditional method and is no less serious for being implicit compared to being explicit in the intelligence analysis. Usually it will be challenging for most intelligence analysts to use, as most are not that mathematically capable and cannot express degree of belief to the precision inferred by the numerical value.

Another feature discussed in the article was the feature of Bayesian method that the analyst does not take the available evidence at face value and then draw conclusions regarding the merits of opposing hypotheses. Zlotnick states, “He rather postulates, by turns, the truth of each hypothesis, addressing himself only to the likelihood that each item of evidence would appear, first under the assumption that one hypothesis is true and then under the assumption that another hypothesis is true. He does not feel called upon to reinforce his self-esteem by reaffirmation of opinions previously put on the record.”

Finally the last feature of Bayesian method is that the analyst comes to a conclusion on evidence given. The analyst does not add the evidence as he or she would typically do to judge its meaning for the final product. The math does the summing up, saying to the analyst that these pieces of evidence equal this conclusion. Research suggests that analysts are better at distinguishing a single item of evidence than at drawing inferences from the evidence as a whole.

Bayesian could possibly complement intelligence analysis when using it for strategic warning analysis due to the fact that it resolves around the odds favoring one hypothesis (say imminent attack) over another hypothesis (no imminent attack). Other ways to test Bayesian in the intelligence analysis field is to test it on international crises from the past. This is done by reviewing evidence from the past and looking at how they made estimates based on their evidence. In the end there must be an assignment of L values and likelihood ratios.

Critique:
Overall this was article an excellent article explaining how intelligence analysts were attempting to incorporate Bayesian into their analysis. Probability theory is quite relevant in the intelligence community. The author did a good job of explaining what Bayesian is and how it could be used for intelligence analysis. While the author did discuss the strengths of using it, he also spoke of its weaknesses. He looked at several ways that it could be used.

Source:

Improving Predictions Using Ensemble Bayesian Model Averaging

Summary and Critique By Claude Bingham

Summary

Because Bayesian models are considered to be strong simulations of scenarios, a group of researchers decided to test whether combining multiple models into an ensemble changed the forecasting accuracy. The researchers pointed out that political scientists rarely use simulations to predict future events, preferring to construct models that attempt to validate theories based on past events. This experiment attempted to show that future events could be tested in a similar way.

To do so, they developed a variation of Bayesian statistics called, Ensemble Bayesian Model Averaging (EMBA), that pools information from multiple forecasts to create an ensemble prediction, similar to a weighted component average. To get the weighted average, a validation period was used to ascertain relative accuracy for each component model. The aim is not to show which model is the most valid, but to show that more model variations gets a more accurate result.

The researchers tested this method in multiple scenarios. The first was a prediction of violent insurgency. Using data, for 29 countries in the 12 calendar months of 2010, three models were constructed. There was a machine learning period from January 1999, to December 2007. Then, the validation period was set from January 2008 to December 2009. The tested period was 2010. One model proved so inaccurate that it was weighted at 0.00. The other two models were rated at .85 and .15. When comparing the results of 2010, the EMBA model reduced error in the prediction by .43 (43%). The EMBA model also showed higher percentage of correct observations and lower average squared deviation of the predicted probability from the true event. This last one means the results of any model calculation for any observation was closer to the actual observation that rival models.

In the example of presidential election forecasts, it was shown that having too many models with high correlations would actually harm the accuracy and validity of the ensemble. While The EMBA was closer on average and had less deviation from actually observed results, it was never the most accurate model. However, it was also never the farthest away.

Finally, two models were used to test accuracy of Supreme Court decision predictions. One was subject matter experts, and the other was an statistical algorithm model based on case factors. When combined, the EMBA outperformed how both individual methods performed separately.


Critique

I appreciated the use of multiple case scenarios with vastly different parameters and outcome types was greatly appreciated. Additionally, the way the researchers pointed out flaws in their method based on the scenario greatly helps plan for statistical defects should someone use this model. I would have liked to see a scenario with one-off characteristics; it would also have been helpful. I do understand that one-off events are hard to organize into standardized variables, but as an intelligence professional, many events that truly matter manifest as one-offs. 



The original research can be viewed here: https://pages.wustl.edu/montgomery/ebma

Bayesian Versus Orthodox Statistics: Which Side Are You On?

Summary and Critique by Michael Pouch

Summary:

This study takes note of psychology and other disciplines benefiting from a set procedures that extract inferences from data. The author of the study, Zoltan Dienes, the purpose of the study is to know if we could be doing the procedure better. Two approaches he compares are orthodox statistics versus the Bayesian approach.  Throughout the article, Dienes breaks down these two into scenarios. First, he presents how hypothesis testing between orthodox statistics differ from Bayesian inference. Second, he shows how Bayesian inference follows from the axioms of probability, which motivate the ‘likelihood principle’ of inference. In addition, he explains how orthodox answers to the scenarios in the test violate the likelihood principle and the axioms of probability. Then he draws a distinction between Bayesian and orthodox approaches to statistics is framed in terms of different notions of rationality. Lastly, he uses the Bayesian approach to enable the most rational inferences from the data.

As the author begins to explain the differences between the two approaches he gives a quick explanation by showing that the orthodox view of sampling is infinite and decision rules can be sharp, while the Bayesian approach treats unknown quantities as probabilistically and the state of the world can always be updated. In general, the orthodox view sees data as a repeatable random sample that has a frequency, where the underlying parameters remain constant during this repeatable process. On the other hand, Bayesian approach observes data from the realized sample, where the parameters are unknown and described probabilistically.

After giving a brief overview of the differences, the author shows how each approach test the axioms of probability through 3 different research scenarios. As he ran each approach, he found that the Bayesian approach is most likely to demand that researchers draw appropriate conclusions from a body of relevant data involving multiple testing. He also identified that the orthodox approach is irrational because different people with the same data and same hypotheses could come to different conclusions.
              
The author then next explains the rationality between the Bayesian and orthodox approaches. He reveals that notion of rationality is about having sufficient justification for one’s beliefs. In addition, if the researcher can assign numerical continuous degrees of justification to beliefs, then the desired requirement can lead to the likelihood principle of inference. With hypothesis testing, the author explains that it violates the likelihood principle, this due to held intuitions we train ourselves with the orthodox method of statistics are irrational toward the key notion of rationality.  The Bayesian approach factors in a connect theory into the data in appropriate ways where it considers an effect size. Bayes factors, but not orthodox statistics, tell us when there is no evidence for a relevant effect and when there is evidence against there being a relevant effect.
             
In conclusion of this study, the author suggests that the Bayesian approach is sufficiently compelling that researchers should be aware of logical foundations of their statistics and make an informed choice between approaches for research questions.

Critique:

The argument that the author lays out is philosophical. He tries to see how researchers can extract inferences better by putting orthodox statistics against the Bayesian approach.  Where Bayesian analysis treats unknown quantities as random variables and where the orthodox treats it as a fixed, the author lays out certain test to show the notional truth behind any sampling model is that is not fixed but random. In the end, The Bayesian reply is twofold. First, by treating the prior distribution as the random variable does not mean that we believe the result is a random variable but rather, it expresses the state of our knowledge about the result. In addition, the Bayesian approach helps us to make inferences while also learning from the data. Despite this, the author did not consider the problems that most Bayesian assessments face. One problem the author did not mention is the choice of prior distributions can be distorted through cognitive bias or little prior information.  Having prior information can help develop a probabilistic result but having noninformative priors not only affects your confidence of the prior information but also your confidence in the result. In other words, people tend to believe results that support their preconceptions and disbelieve results that surprise them.

References:
Dienes, Z. (2011). Bayesian Versus Orthodox Statistics: Which Side Are You On? Perspectives on Psychological Science, 6(3), 274-290

Tuesday, October 24, 2017

Summary of Findings: Yoga (3 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in October 2017 regarding Yoga as an Analytic Modifier specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
Yoga is a philosophy and physical exercise that aims to cultivate mind-body awareness and higher states of consciousness. For analytic technique purposes, it can be used as a modifier. Using breathing, meditation, and modified body posturing, yoga may help strengthen the body while relaxing and focusing the mind. Yoga helps promote blood flow to the brain to promote a feeling of wellbeing, to reduce stress.  
Strengths:
  • Allows the body to repair muscles, ligaments, and tissues
  • Better cognitive performance
  • Improves executive function task performance and mood
  • Strengthens the immune system
  • The mind becomes distracted and helps the person relax
  • Keeps you alert, able to focus, and less irritable
  • If you can convince your team to make the change it is a small, low cost, easy step to increase productivity
  • Could be cost-effective for a company
  • Decreases risk for health conditions

Weaknesses:
  • Requires a physical and mental buy-in regarding concepts
  • Takes a serious time commitment to be fully effective- 60 minutes, 3-4 times a week to see the benefits
  • Not everyone is physically capable of practicing yoga
  • Doing the same routine limits your ability to grow and gain flexibility  
  • Other physical activities could be easier for people to commit to a busy schedule.
  • Many of its postures are athletic, requiring twisting and bending; if not done properly, injury can occur
How-To:
  1. Set aside 30 to 60 minutes two to three times per week
  2. Find a clean, open space to perform the yoga
  3. Breath deeply. In through nose and out through mouth
  4. Move through various postures from a chosen style of Yoga
  5. Concentrate on deep breathing during postures
  6. Change poses, focusing on the breathing
  7. End the yoga with shavasana to feel energized

Application of Technique: The class took an online memory test before participating in a 10 minute yoga exercise using basic yoga postures. After the exercise, the class took a different memory test to assess the differences in cognitive function. Following the second test, the class discussed the pros and cons of yoga compared to other forms of exercise, including, the “flow” state of mind that physical and mental activities can cause.



For Further Information:

Friday, October 20, 2017

Examining the Acute Effects of Hatha Yoga and Mindfulness Meditation on Executive Function and Mood - Kimberley Luu and Peter A. Hall

Summary and Critique by Evan Garfield

Summary
Hatha yoga, the most popular style of yoga in Western society, aims to cultivate mind-body awareness and higher states of consciousness. It involves mindful physical posturing, breathing exercises, and mindfulness meditation. Mindfulness meditation is the practice of non-judgemental observation of present thoughts, emotions, and body sensations with openness and acceptance.

According to the authors, hatha yoga has the potential to improve a variety of cognitive functions including attention, memory, and executive function (EF). EFs are a set of cognitive abilities that allow for self-regulation of thought, emotions, and behaviors. EFs includes inhibition, initiation, working memory, self-control, planning/organization, emotional control, and mental flexibility.
Studies also show that hatha yoga has the potential to improve mood outcomes. The authors explain that hatha yoga is a beneficial treatment for various mood disorders including psychological distress, anxiety, and depression.

The authors emphasize that current literature on yoga is robust and lacks investigation of acute practice. Furthermore, they suggest that cognitive and mood benefits may be separable. Accordingly, they designed an experiment to compare the acute effects of mindfully practiced hatha yoga and mindfulness meditation on its own on cognitive and mood outcomes.

Findings revealed that 25 min of hatha yoga and mindfulness meditation significantly improved EF task performance and total mood. Although hatha yoga presented a greater overall effect, improvements did not differ significantly from each other. Yoga's advantage was driven by complex physical postures that help increase attentional processes. According to the authors, the improved effects emerged following a 10 minute delay. These results were consistent with findings of other studies on acute exercise, in which effects emerged during the 11 to 20 minute post-exercise period. The authors suggest the meditation-induced sedative effects of yoga and mindfulness meditation may need time to subside before cognitive benefits are identifiable.


Critique
This study gave some interesting insight into the benefits of yoga and mindfulness mediation on cognitive performance and mood. Although the results did not differ significantly, it was interesting to see that yoga had a stronger effect than meditation by itself, attributed to complex physical postures driving attentional processes. Furthermore,  while the authors reveal that yoga and mindfulness meditation improved EF task performance, they fail to elaborate if specific EFs(inhibition, initiation, working memory, self-control, planning/organization, emotional control, mental flexibility) were improved more than others. It would be interesting to conduct further research focusing more narrowly on these attributes. Overall, both hatha yoga and mindfulness mediation appear to be valuable exercises for employers across all sectors to improve morale, productivity, cohesion, organization, and planning within their organizations.


Source: https://www.researchgate.net/publication/311963232_Examining_the_Acute_Effects_of_Hatha_Yoga_
and_Mindfulness_Meditation_on_Executive_Function_and_Mood

The Effects of Yoga on Stress, Stress Adaptation, and Heart Rate Variability Among Mental Health Professionals--A Randomized Controlled Trial

By Samuel Farnan

Summary:


A group of researches set out to measure the effect Yoga would have on a group of mental health professionals. They argue the need for this study is due to the increasing demands by the general public for mental health therapy. More so, the amount of emotional stress that mental health professionals are subjected to and more importantly, expected to overcome is also increasing. These professions include psychologist, psychiatrists, social workers, and occupational therapists. 

The research design utilized 60 mental health professionals that at the time of the study, were not involved in any regular exercise program. 30 of the mental professionals did yoga once a week for 60 minutes and 30 did not, with the test span lasting a total of 12 weeks. Results of work-related stress and stress adaptation were measured via biofeedback monitors (objective) and personal surveys (subjective). The biofeedback monitors measured various nerve activity while questions for the survey included "Do you feel overloaded?" and "Do you experience difficulty in getting along with colleagues?". These measurements were calibrated on a 60-point stress scale.

Overall, the participants who did yoga displayed increased adaptation to stress and lowered stress that was directly related to their work. In a more specific analysis of co-variance (ANCOVA) the yoga group showed significant change in work-related stress and autonomic nerve activity, but not actual stress adaptation. The researchers propose that yoga can offer physiological and autonomic balance to manage stress, but not the actual skills of stress adaptation. Furthermore, the researchers proposed further study should include more yoga--two to three times of this study-- to determine an optimal range of yoga practice for mental health professionals.

Critique:


Although the benefits of yoga are well known at this point, I feel this study applies quite well to intelligence professionals. Like these mental health professionals, intelligence professionals are expected to make the most objective decisions possible, eliminating bias and emotion often under tight deadlines and stressful conditions. However, this study could've done better in some areas. The researchers took 60 mental health professionals that were not on any physical exercise regimen prior to the 12-week study. Is yoga more effective on stress management than running, lifting, swimming, etc.? I would enjoy seeing yoga compared to other physical activities, especially swimming, as both utilize and depend on breathing sequences along with the reduced stimuli of both activities. 



The Effects of Yoga on Mood in Psychiatric Inpatients



Summary and critique by: Kevin Muvunyi

Summary:
A group of researchers conducted a study in 2005 to determine the effects of practicing yoga, a widely renowned relaxation and meditation method in the world, on the mood of 113 inpatients at the New Hampshire Hospital. Prior to the start of the research exercise, inpatients were directed to answer a Profile of Mood States (POMS) questionnaire. This questionnaire which is a measure of the six major negative emotion factors was also handed out to be completed by the inpatients at the end of each yoga session and this for a total period of ten days. The inpatient group used in the study was comprised of a total of 59 women and 52 men.

As a result of the study, the researchers were able to observe that practicing yoga demonstrated a positive effect on five of the six major negative emotion factors on the POMS. Namely the tension-anxiety, depression-dejection, anger-hostility, fatigue-inertia, and confusion-bewilderment emotional elements. On the other hand, participating in yoga sessions showcased no positive effect on the sixth PMOS factor, which is vigor-activity. Moreover, the researchers noted that the participation of an individual in more yoga classes than his peers had no substantial consequence on his overall mood.

  
Globally the initiators of the research concluded that yoga could prove to be an efficient technique to reduce stress levels and mental illness symptoms amongst patients in highly restrictive and controlled areas such as hospitals based on their observations. Nonetheless, they conceded that due to the uncontrolled nature of the study, further research was required to validate their findings. The researchers also pointed to the possibility of their study being inaccurate based on the premise that the inpatients completed the POMS forms with a prior intent of satisfying the perceived expectations of those conducting the research.

Critique: 

Overall the study was well designed with a sufficient sample size to make substantial observations. Nonetheless, the uncontrolled nature of the experiment suggests that the findings of the research exercise are inaccurate. Further research, should explore the possibility of introducing a control group in the experiment, and also seek to explain the reason behind the inefficacy of the yoga technique in regards to improving the emotional factor of “vigor” in a mentally ill individual.

Source: http://eds.a.ebscohost.com.ezproxy.mercyhurst.edu/ehost/pdfviewer/pdfviewer?vid=8&sid=4a895aa1-22dd-437e-9120-15c811c5272d%40sessionmgr4008