The Science Behind Lumosity

Summary:

This article counters the idea that the core aspects of cognitive processing are determined and fixed at a young age leaving little to no means of improvement and individuals born with strong cognitive capacities through genetics and early development had an advantage throughout their lives while individuals not encompassing these capabilities had no room for improvement.  The Lumosity process suggests that aside from genetics, the right type of stimulation and activity, at any age, the brain can change and remodel itself to become more efficient and effective in the areas of processing information, paying attention, remembering, thinking creatively, and solving novel problems.  The main areas that make brain training and Lumosity effective include targeting, adaptively, novelty, engagement, and completeness.   

The brain’s ability to reshape itself is called neuroplasticity.   To exemplify the behavioral changes in the brain the study gives an example of a test potential taxi drivers must pass in London, a rigorous exam that tests the point-to-point routes in the city refereed to “The Knowledge."  After this exam and continuous studying, they found major structural changes in the brain including differences in the size and shape of crucial brain structures in taxi drivers relative to control subjects.  The hippocampus, a brain structure critically involved in memory and navigation, was larger in the taxi drivers over the control group. 

Other activities such as video gamers performed better in visual attention than non-players.  Furthermore, a study asking non-players to play an action video games intensively over several weeks found their visual attention capacities improved.   The article then goes on to discuss Lumoncity' s ability to provide effective brain training for all ages including older adults, children (incorporating those with ADHD), and young adults. 

Finally, the study presents a scientific framework of Lumosity.  It talks about a 2006 study that evaluated the effect of Lumosity training on cognition in normal, healthy adults that included 23 participants with a mean age of 54.  These individuals divided into a control group and a group that received Lumosity training 20 minutes per day once a day for five weeks.  The study found that participants improved on the games that they played as well the training gains transferred to measures of cognitive performance that were not directly trained.  The participants did not only learn strategies to play the games but the underlying brain mechanisms fundamentally changed after training. 

Critique:

Lumos Labs wrote this article, the creator of Lumosity therefore, a degree of bias toward their product is likely to exist.  While this is true, the article cites multiple scholarly articles to back up the claims.   That said, considering Lumosity is the most popular method of brain training today, given my limited knowledge on the topic I thought this article would provide a useful overview and description of not only brain training but Lumisuty’s approach.  The article also includes a variety of other credible studies published by leading scholars that study brain training supporting Lumosity’s claims. 

Understandably, as a business trying to make money, the article had promotional aspects to it.  There were no downsides mentioned in terms of brain training or Lumosity.  Additionally, while this article looked at Lumosity and other key studies on brain training, throughout the research there was not an article tailored to compare and contrast the many types of brain training for consumers to decide which method or product works the best or what product would provide them the most benefit. 

Overall, this article comprises a large sample of the most compelling research done on brain training.  This aspect makes it a very useful reference providing a short synopsis of these studies under the sub heading Broad and Growing Base of Evidence that Cognitive Training Works as well as throughout the article.

Source:

Hardy, J., & Scanlon, M. (2009). The Science behind lumosity. Lumosity. 

Summary of Findings (White Team): Visual Analytics (4.3 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 8 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in April 2013 regarding Visual Analytics specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
Visual analytics is a broad category of modifiers that refers to the visualization of data in a manner that simplifies the comprehension of data and information and aids in the analysis and communication of results.

Strengths:
1. Provides visuals to identify patterns.
2. Can provide a tangible three-dimensional object to physically hold.
3. Provide an effective way of presenting intelligence to decision makers.
4. Provide more depth to the analytical product through referencing to the visual.
5. Has the potential to provide an interactive means to display the estimate or information.
6. Can display relationships visually that might have been overlooked with the utilization of other methods.  
7. Output can be automated.

Weaknesses:
1. Can be difficult to make it understandable for the consumer or decision-maker.
2. This modifier is not easily defined.
3. Not as helpful for individuals who do not learn effectively visually.

Step by Step Action:
1. Be aware of your decision-maker’s needs and preferences for the product.
2. Determine which form of a visual is most suited to your product and the decision-maker.
3. Make your visuals simple and easy to understand for the decision-maker.

Exercise:
As a class we were given the question to analyze how many Jelly Belly jelly beans would fill a container 5 inches long, 2 inches tall, and 3 inches wide. We were asked as a class three separate times to estimate how many jelly beans would fit into the container. Additionally, we had to give an estimate of how confident we were on each estimate from Activities 1-3. Activity 1 an estimate was made just based on knowing the size of the container and how one perceives the relative size of a Jelly Belly jelly bean. Activity 2 an estimate was made by viewing a picture of the jelly beans in the container. Activity 3 an estimate was calculated by holding the actual container filled with the jelly beans.  

The results as a class were interesting to see how each individual determined their analysis. For some of the class, viewing the container in two different levels of visualization forms increased their level of confidence on their estimate. For other participants in the class their level of confidence in their estimates did not change during each of the activities, but their estimates changed when more visuals were presented. Moreover, it was interesting to see during the first activity many participants in the class tried to draw out the probable size of the container or tried to visualize the size of the container with their hands to make their estimate.

Summary of Findings (Green Team): Visual Analytics (4 out of 5 Stars)

Visual Analytics

Green Team

Rating (4 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 8 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in April 2013 regarding Visual Analytics specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Visual analytics is a collection of modifiers and methods used to improve the analysis of information, as well as representation to increase the effectiveness of the decision making process. Both qualitative and quantitative information is depicted through the modifiers and methods.  The visual representation of the information should increase comprehension of the material in addition to appealing to the viewer’s aesthetics.  

Strengths:

• Decreases the level of ambiguity in conveying information
• Supports people who are visually oriented
• Gains interest of the decision maker much more than a written element
• Has the potential to increase analytic confidence

Weaknesses:

• Not everything has the possibility of integrating a visual component
• Visual analytic products can introduce bias into an analysis
• Not necessarily intuitive to an outside party, especially since there is a different level of input between those that created the visuals and those the information is being presented to
• Does not support people who are not visually oriented
  

How-To:

1. Find a topic that is capable of having some kind of visual component
2. Compare and contrast various ways in which a visual component can be integrated (ex: graphs, charts, information arranged in a particular fashion, etc.)
3. Select the visual component that you feel best communicates the idea while being mindful of who the consumer is or decision maker the visual component is geared towards

Personal Application of Technique:

A written description of the dimensions of the container were provided to individuals.  The class was asked to estimate the number of jelly-beans that were in the container along with recording their level of confidence in that estimate.  In the next stage, a static image of the container filled with jelly-beans was presented to the individuals.  A dime was used to provide a size comparison to the image of the container with the jelly-beans.  Following another estimate and confidence level, the individuals were given the container with the jelly-beans to hold and look at in a three-dimensional manner.  The participants were then asked to give another estimate and confidence level.

The introduction of visual and tangible elements to the estimate generally increased the level of confidence in three of the five participants. One participant raised their confidence only after being presented with the three-dimensional, interactive visual aid. The other two raised their confidence level after the introduction of the 2D, static image. All participants changed their estimates each time a visual was presented, getting closer to the actual number. Some changed drastically (originally estimating 900+, then decreasing the estimate to 350, actually 353), while others remained in the same ballpark but improved their estimate.

Rating: 4 of 5 stars

VAC Views - May 2008

This document titled VAC Views (May II, 2008) written by the Visual Analytics Center and covers a wide range of topics related, but not limited to, integrating visual analytics to enhance learning.  This group previously released these documents biannually and covered many different kinds of topics related to visual analytics.

Summary:

For this document I chose to highlight one particular article of interest to me and what I feel was the most relevant to this class: Active Products: Composition and Dissemination of Smart Analytic Reports.  This report states that one of the biggest problems for analysts is utilizing visualization effectively as part of the communication process with decision makers.  This clearly indicates that not all decision makers are the same.  The author breaks it down to production ("the act of composing reports"), presentation ("the outward form these reports take"), and dissemination ("how the content of these reports is shared across and organization").  The group proposes a new product they are coming out with that is a 'smart report'.  What this means is that the reports have to be made with the idea that they can be taken apart and put back together to give to anyone that may need to use them in an effective manner.  The data must be well sourced along with clearly stated operations the analyst used to acquire the data and process it as intelligence.  Confidence must be also clearly stated and updated as necessary.  Third, the reports must have some interactive capabilities such as a live visual component that engages the reader.  This can put the reader in the analyst's shoes and allow them to see how the analysts see.  Lastly, the products must be customizable to as many groups as they need to be.  Length and classification, among other categories, are examples of criteria these reports must be able to alter.

One specific tool the National Visualization and Analytics Center (NVAC) proposes is a snippet widget that allows the writer of the report to collect information at the time of research that may be in the form of evidence or other important information.  This also includes how the information was discovered.  These snippets are then combined together for the report to make a coherent argument or story.  Instead of having to retrace and reanalyze the information, this snippet widget will free up time that can be spent on organizing and refining the report.  Additionally, the reports generated from the snippet widget and the refining process can be placed into style sheets.  These style sheets allow for the document to be easily  transferred between different forms of reports including websites, analytic reports, or even blog sites.  These can also be edited by others (if the writer wishes) in order to improve the document or adjust information.  This eliminates the time spent making each document tailored to a specific person in a specific style.

Critique:

I really liked reading about the articles in this edition, as well as skimming other editions.  I think a lot of the products and articles they put forth are easily understandable and make great points about how visual analytics is improving.  This article in particular really speaks to what we hope to do here at Mercyhurst University (or are at least trying to do).  It's about making a report that a decision maker wants to read and doing it in a timely manner.  I think this is an excellent idea and could really help improve the report writing and publication process of reports.  Additionally, this report briefly mentions the four criteria for creating these new reports.  The third requirement- making products interactive- really emphasizes what I feel is the future of report writing.  Decision makers are not going to just want to read a report- they are going to want to visually see how different factors are working or perhaps even receive the document in a video news bulletin form.  By having a tool that help integrate all of these specifications in a timely fashion is an excellent research endeavor to tackle.

One of the drawbacks mentioned is that there needs to be a research community established that is willing to help improve documents when requested.  I agree that this is a drawback, however, the idea of having a document that is easily convertible and accurately sourced without much additional effort is a huge value in itself.  To be able to write a report and never have to create your own presentation for it, but instead just tweak an already made report would save a lot of time.

Lastly, when I first began to read this article I figured it would focus a lot on how to make charts and figures look more appealing.  I think this is also something that needs to be worked on, however, a report that can convert itself into a different type of report while maintaining citations is bigger time saver than adding different graphs.

Source:
May II, R. (2008). VAC Views - May 2008. Retrieved from http://readthis.pnl.gov/marketsource/readthis/B3065_not_print_quality.pdf

Intelligent Visualization and Information Presentation for Civil Crisis Management

Adrienko & Adrienko's paper "Intelligent Visualisation [sic] and Information Presentation for Civil Crisis Management" describes research conducted as part of the EU-funded project OASIS to develop methods for effective visualization support for situation analysis and management during times of crises, namely through the "Situation Manager" software module. The authors cite the major goals of the research are "to reduce the information load of the analyst, decision maker, or information recipient without omission of anything important and to ensure quick and accurate comprehending of the information" (Adrienko & Adrienko, 2007, p.889).

Summary:
The research behind "Situation Manager" aims to create a generic crisis management system to support response and rescue operations in the event of large-scale disasters. Like intelligence products, visual analytic products should be deliverable in a timely fashion, while the information is relevant, and presented in a way that it is easily understood and therefore can be used by decision-makers. The authors suggest intelligence visualization requires reducing the information load on the recipient, display choices and designs that ensure quick and accurate recognition of meaning, and account for the characteristics of the medium used to view the information.

The authors suggest that intelligent visualization is used for two different purposes: to support the work of an analyst, planner, or decision maker, or to build an information presentation to send to a specific recipient.  It is unclear if the authors see these purposes as mutually exclusive, though it is unlikely. In the development of intelligent visualization, expert knowledge is needed in the emergency management domain ontology, generic roles involved in emergency situations and their information needs, and techniques and methods to manipulate and organize different types of data. The research uses literature on crisis management as a source of domain-specific knowledge and literature on data analysis, graphic design, and geographic visualization as domain-independent knowledge to create an expert system.

"Situation Manager," still in its early stages at the time of this article's publication, is a scalable vector graphics presentation that is interactive and includes the emergency management expert information and a problem-specific user interface. The system takes information about the crisis situation and the territory affected by the situation, essentially answering the what and where questions. The where aspect includes an impact zone which can be supplied or self-created. The software module then identifies the hazardous agents involved in the situation based on preexisting knowledge and potential secondary hazardous events caused by the first crisis event. This helps to locate objects or resources that need to be saved or protected in the impact zone to prevent further damage, including an estimate of the number of people living in the zone based on census data. If the object is people, the Situation Manager detects if people in the impact zone can escape danger without outside intervention. The module displays objects and dangers by degree of how critical each is through the sizes of symbols used on the maps, larger symbols representing a more critical object at the current time.

Critique:  
This research is particularly applicable to the intelligence field considering the blatant similarities between what the authors conclude are requirements for intelligent visualizations and the requirements we at Mercyhurst accept as necessary for an analytic/intelligence product. When reading the article I could specifically relate it to the work a classmate is doing to identify the potential for nuclear exposure caused by earthquakes in Iran.

The article itself is only a description of research being conducted, rather than methodologically reporting the findings and methods of the research. While the article suggests different sources of information used to create the expert system for visualization for crisis management, it would strengthen the presentation of the research to formally address all areas of data collection. Still, the authors do a good job of explaining the research in developing the "Situation Manger" in general terms and use good examples to communicate the use of such a tool. Though the module was not complete at the time of the article's publication, the authors provide a roadmap of implementation upon completion and suggest likely real-world applications.

It is interesting to me that the article does not address why visualization is a useful communication tool for decision-makers in general, not even referring to the extant literature on the subject which is extensive. Further, because the module was not complete at the time of publication, there is no record of successful implementation or feedback by users. Further analysis of this type of visual analysis and data visualization is necessary to strengthen the argument for the Situation Manager module.

Source:
Andrienko, N., and Andrienko, G. (2007). Intelligent Visualisation [sic] and Information Presentation for Civil Crisis Management. Transactions in GIS 11(6), 889-909. doi:10.1111/j.1467-9671.2007.01078.x

Provenance of Intelligence Analysis Using Visual Analytics

Summary:

Intelligence agencies often retrieve information from multiple sources to anticipate and counter terrorist attacks. Often difficulties of organizing this information arise when dealing with large quantities of dynamically changing information. Interactive visuals tools allow for effectively analyzing data and providing faster insight. Post Cold War centralized terrorist organizations have been replaced by decentralized terrorist cells that continually reinvent threats. In this case, visual analytics in intelligence is extremely useful for identifying behaviors of terrorist groups. For example, some of the advantages of using visual tools to study behaviors of a network of individuals include identifying key players of the complex network, identify emerging themes, indicate geo-spatial relationships, and show correlations across multiple parameters.  

During intelligence analysis, raw information processed through multiple stages is transformed into actionable intelligence. According to the authors, processing a large amount of raw material requires the use of visual tools to track the transformation. Both evolving and complete analysis should involve a review of the analytic process including tracking the sources of data and their reliability, reviewing background knowledge of the scenario in question, and analysts assumptions. Also known as provenance information, information that is traceable is valuable when assessing the plausibility of conclusions. Also a framework of permission management can be applied to visual tools in an effort to manage transparency among analysis with differing security clearances. The authors’ approach to traceability includes three categories; data level, analysis products, and reasoning products. The framework is based on products present in any analytic workflow. However, a number of challenges exist in tracing provenance during intelligence analysis. The authors suggest another framework called provenance reasoning workspace that comprise of three spaces to reduce these challenges. The spaces include a data space, a computation space, and a reasoning space.                              

Critique:

Although this article was short, it was not an easy read. The article had a few sentence fragments and run on sentences. I had to read some sentences a number of times to understand the content. The authors focus was on using visual tools to keep track of the process of collecting information to analyzing intelligence as oppose to using it as a tool that enhances the presentation of the final product to the decision makers. Most of the article focused on providing advantages of using visual tools to trace the transformation of information to intelligence followed by frameworks. The information on the article needed to be explained in more than two pages. The authors’ framework for tracing the transformation of information into intelligence is not very descriptive. An example or a methodology section testing the frameworks may have shown the applicability of the frameworks.   

This reminds me of the article that I summarized about decision trees. Decision tress may not trace the process of gathering and analyzing intelligence, but it is a tool that can visually organize data for the analyst’s use rather than the decision maker’s. The authors’ frameworks are prone to similar challenges of incorporating decision trees to intelligence process. Similar to decision trees and contrary to the authors’ view, the provenance frameworks may not be able to handle complex scenarios. It seems quite tedious and requires the use of two different frameworks. Organizing data at each phase will require the use of multiple tools. It also requires the analysts to update each step when incorporating new evidence. I’m unsure of the benefit of tracing the process of creating intelligence from the beginning to the end in a visual manner.

 
Source:
Wong, W., Xu, K., & Attfield, S. (n.d.). Provenance for intelligence analysis using visual analytics. 
         Retrieved from http://eprints.mdx.ac.uk/8415/1/intelligence-analysis-provenance1.pdf

Interactive Dynamics for Visual Analysis

Summary:
In the article Interactive Dynamics for Visual Analysis, Jeffrey Heer and Ben Shneiderman (2012) provide a taxonomic guide for analysts, researchers and other professionals to create visual analysis tools. They discuss the usefulness of visualizing data for comprehension in saying that "by mapping data attributes to visual properties, ... visualization designers leverage perceptual skills to help users discern and interpret patterns." The authors also stress the importance of ensuring that the visuals are appropriate and intelligible for the consumer.

The authors describe three dynamics for visual analysis, each of which include examples of task types or steps that fit into those descriptions. The three dynamics are Data and View Specification, View Manipulation, and Process and Provenance. Data and View Specification involve determining which data is to be shown and visualized with programs such as Microsoft Excel. Then it involves the filtering of data which shifts the focus among the different data subsets to isolate specific categories of values. Sorting the data can show surface trends and clusters and organize data according to a unit of analysis. The following image shows a more complex form of a matrix-based visualization of a social network.

The first matrix plot shows a social network when people are sorted alphabetically. The second plot shows a reordering by node degrees resulting in more structure and the third plot is permutated by network connectivity, showing underlying clusters of communities. The final step is to derive new attributes from existing values when input data is insufficient.

The second dynamic is View Manipulation which consists of highlighting patterns, investigation of hypotheses and revealing additional details. Selection allows for pointing to an item of interest, for example, dragging along axes to "create interactive selections that highlight automobiles with low weight and high mileage." Navigating is determined by where the analyst begins, such as in a crime map that depicts crime activity by time and region. Coordinating allows the analyst to see multiple coordinated views at once which can facilitate comparison. This can be done in histograms, maps or network diagrams. The following image shows a complex patchwork of interlinked tables, plots and maps to analyze outcomes of elections in Michigan.

The image shows a combination of tables, plots and maps. The final step, organization, involves arranging visualization views, legends and controls for more simplified viewing.

The final dynamic is Process and Provenance which involves the actual interpretation of data. Recording involves chronicling and visualizing analysts' interaction histories in both a chronological and sequential fashion. Annotation includes recording, organizing and communicating insights gained during analysis. Sharing involves the accumulation of multiple analyses and interpretations derived from several people and the dissemination of results. Guiding is the final step and includes developing new strategies to guide newcomers.


Critique:
This article was very effective in both showing many ways of visualizing data and also the importance of visualization in comprehension and facilitation of analysis. The inclusion of some well-known applications such as Google searches and crime maps and their many examples were beneficial in the authors' explanation of the taxonomy for those readers who don't have extensive experience with other specialized software or the field. It was also important that the authors distinguished between what may be visually intriguing but don't have much real-world application.

Some of the features explained within each dynamic were directly applicable to the intelligence field or anyone conducting an analysis of data, such as the investigation of financial markets or terrorists networks. I can foresee this field being a new avenue to explore for more effective communication between analysts and decision-makers. Analysts often learn that decision-makers prefer concise, clear and preferably visual information but may not know an effective manner to convey such information. This article provides a basic but helpful overview in how to go about visualizing data.


Source:
Heer, J & Shneiderman, B. (2012). Interactive Dynamics for Visual Analysis. Communications of the ACM, 55(4), 45-54. doi:10.1145/2133806.2133821

Applied Visual Analytics for Economic Decision-Making

Summary:

Savikhin et al. (2008) utilize the application of visual analytics to improve an individuals' economic decision-making skills.   The authors investigated the application of visual analytics to common problems noticed in economics, the winner’s and loser’s curse.   The winner’s curse is the individual who tends to overpay for certain items or services, either the individual is worse off for buying the product or service, or the value of the asset is less than the bidder perceived.  The loser’s curse is when an individual pursues an asset that is below their profit-maximizing bid, or a competing entity attains the bid.   The main problems apparent is that economists are unable to see the potential for creating a business strategy that is able to maximize profit, with most economists are unable to consider all the information that could guide these decisions.   Thus, the authors apply visual analytics to improve the decision-making abilities in both winner’s and loser’s curse situations.   The hypothesis by Sayikhin et al. (2008) for their study was the subjects who participated in their interactive visual analysis study would bid closer to the profit maximization decision as opposed to those who participated in simple visual or tabular studies. 

Sayikhin et al. (2008) conducted six different treatment groups, 3 for winner’s curse scenarios and 3 for loser’s curse scenarios.  The three different visual aids the participants looked at to help with their decision-making were an interactive visual analytic model, a simple visual, and a tabular table.  Each subject in the experiment acted independently from other subjects in the study.   All were given the scenario of being a decision-maker who had to decide how much to bid for a company.   Participants were given a possible data range that they could bid for each company.  Decisions on how much to bid were conducted on a computer generated program that would randomly decide the value of the company and display the three different types of graphics.  Over the course of the experiment the participants switched between the three types of visual aids and would base their bid value off their interpretation of what the visuals portrayed.   In each of the three different visual representations individuals were given 30 different opportunities to bid on various companies. 

Overall, subjects that were given the interactive visual analytics treatment learned what the best bid/optimal solution would be more often as compared to those individuals who were given simple visual or tabular representation of the bidding information.  Moreover, for both the winner’s and loser’s curse scenario groups, the periods of using interactive visual analytics outperformed subjects given the other visual treatments.   Overall, results were statistically significant in this regard. Each increased usage of the interactive visual analytics model allowed the participants to learn from past decisions on bids and allowed these individuals to make more optimal bids as opposed to other participants who received the other two visual treatments.  It is also important to note that even a simple visual aide provided more effective decision making capabilities as opposed to viewing information with tabular formed displays.

Critique: 

I found that this study was useful as it provided a way in which to help individuals within the business realm to make more efficient decisions by analyzing their situation with interactive visual aids.  It is important to avow that this study seems to suggest the usefulness of showing information visually to overcome cognitive thinking judgments from decision-making and improve learning capabilities.   Moreover, it would be interesting for a future study to demonstrate how interactive visuals seem to engage our thinking more than just a simple visual does.  One limit of this study was that the sample size was small, so it would be interesting to conduct this study over a much larger sample size to replicate the results.  Another limitation of this study other than sample size was that the authors only looked at bidding patterns in winner’s curse or loser’s curse scenarios, not any other economic conditions.  Even though this is a topic that would come up often within the business environment, it would be interesting to see what other areas in business decision-making scenarios would interactive visual analytics improve the process of decision-making.   I would hypothesize that interactive visual analytics would be able to be applied to multiple areas in the business realm, especially for those individuals who learn more effectively visually.  

Source: Savikhin, A.,Maciejewski, R., & Ebert, D.S. (2008). Applied visual analytics for economic decision-making. IEEE Symposium on Visual Analytics Science and Technology, 107-114.  Retrieved from https://www.bioinformatics.purdue.edu/discoverypark/vaccine/assets/pdfs/publications/pdf/Applied%20Visual%20Analytics%20for%20Economic%20Decision-Making.pdf.

Empirical Studies of Information Visualization: A Meta-Analysis

Summary:

The study Empirical Studies of Information Visualization: A Meta-Analysis by Chaomei Chen and Yue Yu provides a meta-analysis on a variety of empirical studies of information visualization.  The intent of the research is to conduct a meta-analysis on this topic in order to capture the theories and practices in empirical examinations of information visualization. The analysis focuses on three areas of information visualization including users, tasks, and tools.  As a meta-analysis, this article provides a simplified description and displays the underlying relations in the large amount of convoluted, contradictory, and confusing information often found in the literature.  

The article first provides an overview of the meta-analytical method and selection for studies used, then a subjective review of the studies is presented follow by identifying the most commonly used hypothesis, independent variables, and dependent variables, this is followed by the results of the study.  The research includes experimental studies with independent variables that comprise a relationship to one of the three contextual variables (users, tasks, and tools).  The two types of dependent variables used are accuracy and efficiency measures. 

The study’s results come in two parts looking at both users and tools.  Each section compares the empirical findings of individual studies and are synthesized in terms of effect sizes and significance levels.  The study found that users with strong cognitive abilities will benefit significantly more from visual-spatial interfaces than those with weaker cognitive abilities.  The study found that users with stronger cognitive abilities will perform more efficiently than users with weaker cognitive abilities while using visualization. Additionally, the study displayed that visual-spatial information-retrieval interfaces will enable users to perform better than traditional retrieval interfaces.  Finally, the study stated that users using visualization interfaces in information retrieval will perform more efficiently than those using a none visualization interface.  The following list the major, all encompassing conclusions of the study.

                1. Empirical studies of information visualization are diverse and applying meta-analysis methods is difficult.

                2. Future studies would benefit from systematically investigating individual differences, including a  variety of cognitive abilities systematically.

                3. When users displayed the same level of cognitive abilities they tended to perform better with simpler visual-spatial interfaces.

                4. The combined effect size of visualization in not statistically significant. A larger homogeneous sample of studies is necessary for conclusive results.

Critique:

This meta-analysis is especially helpful due to the increasing amount of literate on the topic of visualization.  Although its finding were not significant it provided a very effective start in providing an overview of the current literature on visualization.  As technology progresses this study will be one to build upon.

Considering the study came out when visualization was in its infancy, it is understandable that a minimal amount of articles were available.  Although, if the authors would have slightly expanded their criteria the study would have provided a better overview of visualization overall.  Furthermore, only five studies tested the effects of visualization on accuracy.  To strengthen the argument the authors should have considered increasing the amount of articles used under this category.  Additionally only three studies tested the efficiency of visualization. The same critique applies here in that more sources would have greatly benefited the analysis as well as the argument overall.  Again, by expanding the criteria for an article to be used in the study as well as the categories tested in the study more articles may have been available.    

Overall, this study provided a very useful synthesis of visualization and its benefits.  Expanding upon the approach by conducting a similar study in terms of visualization today would provide an interesting comparison and an overview of its progression.   

Source:

Chen, C. & Yue, Y. Empirical studies of information visualization: a meta-analysis. Int. J. Human-Computer Studies. 53(5), 851-866. Retrieved from: http://www.sciencedirect.com/science/article/pii/S1071581900904221.

Geographical Information Systems–Based Marketing Decisions:Effects of Alternative Visualizations on Decision Quality

Ana-Marija Ozimec, Martin Natter, & Thomas Reutterer conducted a study to examine the effectiveness of different quantitative symbolization methods on maps. The researchers wanted to know which type of visualization method was the most effective with decision makers.

The symbolization styles they examined  were size-based circles and bars, shadings (value), pure (circles, bars, and shadings), combined (shadings and distortion), circles, and bars. To measure the effectiveness of these symbolization styles, the researchers examined decision accuracy, decision confidence, decision efficiency, and perceived ease of task. Below is a chart of their findings.  

The researchers found that using circles was the most effective visualization methods for decision makers. Using all four measures of effectiveness, circle symbolization was the best performing one. Other results showed the combined shadings and distortion ranked second in decision accuracy, but ranked lowest in decision confidence. Value shadings ranked lowest in decision efficiency while shadings and bars ranked lowest in perceived ease of task.

Critique 

This study has direct implications to intelligence analysts. This study found that circle symbolization of quantitative information is the most effective symbolization tool for decision makers. As intelligence analysts, it is our job to continue to improve communication between ourselves and our decision makers. The results of this study can help us do that. Since GIS data can be used by all types of intelligence (national security, law enforcement, and competitive), this study can be applied across all fields as well.  

The only criticism I have is that circles as a form of quantitative symbolization may not be applicable to all types of scenarios or quantitative data. Though at this moment I cannot think of any, there might be a situation where circles are not the best form of visualization. 

Source: Ozimec, A., Natter, M., & Reutterer, T. (2010). Geographical Information Systems–Based Marketing Decisions:Effects of Alternative Visualizations on Decision Quality. Journal of Marketing, 74 (6), 94-110. Retrieved from http://ehis.ebscohost.com/eds/pdfviewer/pdfviewer?sid=8fe30abf-7ab8-4068-be2c-26bac62fd9d7%40sessionmgr4&vid=2&hid=4# 

Investigative Visual Analysis of Global Terrorism

Summary:
The application of visual analytics to global terrorism.  The authors, Miler, Smarick, Ribarsky and Chang, used an existing database with information on terrorist organizations to apply visual methods.  The Global Terrorism Database (GTD) contains information on both domestic and international terrorist organizations.  The authors applied their visual analytic system to this existing database to look at the five W's (who, what, where, when and why) of terrorist organizations in a manner that is easier for decision makers to understand.

Prior to this tool there were typically two groups of visual analytics: social network analysis and geo-temporal visualizations. The system implemented by the authors attempts to combine both social network analysis and geo-temporal visualization  This tool has a number of layers that can be activated to look at the various elements of terrorist organizations.  There are levels that show the location of attacks, which can be detailed on to see the specifics of what took place at that location. Through the different layers various elements can be visually depicted, which makes it easier to understand the data that is present.

The authors indicated that there are three types of individuals who typically go to the GTD website: the general public, investigative analysis, and terrorism experts.  Through the visual analytics tool, individuals with varying levels of exposure to the subject matter can gain a significant level of understanding of the material.  When the system was used by individuals in various organizations, they were all interested in applying the method to the various fields they were in.

Critique:
One element the authors identified in their conclusion was that there were certain elements that could be enhanced overtime.  For example, there were instances of over-plotting data and geographic lines, which made it difficult to look at.  Overall, it appears that this method is extremely useful to apply to existing data. Not only does it analyze various elements, but it also increases the ease of communicating with decision makers, as well as decrease the ambiguity that may be present in large data sets.  This method is certainly something that should be incorporated when possible and certainly enhances the distribution element of information.

Wang, X., Miller, E., Smarick, K., Ribarsky, W., & Chang, R. (2008). Investigate visual analysis of global terrorism. Computer Graphics Forum, 27(3), 919-26. Retrieved from: http://ehis.ebscohost.com/ehost/pdfviewer/pdfviewer?sid=74559524-5a41-4427-a79e-c5e026f76e72%40sessionmgr110&vid=3&hid=17

Summary of Findings (Green Team): Bayesian Analysis (3.75 out of 5 Stars)

Game Theory

Bayesian Analysis

Rating (3.75 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 8 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in April 2013 regarding Bayesian Analysis specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Bayesian analysis, developed by Thomas Bayes, is a statistical approach that deals with using prior knowledge as well as updated information, separating the Bayesian approach from frequentist statistics. Bayesian analysis makes the analysis process iterative, allowing for more information to be added as it is learned or deemed relevant. According to Hubbard (2010), Bayesian theorem is a relationship of probabilities and conditional probabilities, or the chance of something given a certain condition (p.178-79). Bayesian analysis allows analysts to calculate probabilities or make estimates in terms of certain base assumptions as well as new developments. This is especially important in the intelligence field as analysts should be able to include new signals or indicators into their analysis to update estimates, further reducing their levels of uncertainty.   

Strengths:

• Most helpful in situations in which your initial estimate is at either end of the spectrum
• Helps analysts to combat emotional/irrational estimates
• Particularly effective when analyzed in relation to the probability that was present prior to the addition of the newest piece of evidence.
• Has the potential to be applied across multiple disciplines, as evidenced more frequently in recent academic journals

Weaknesses:

• Complex, difficult to learn without a statistics background
• The volume of evidence makes it difficult to continuously evaluate the information
• No set way to decide what constitutes evidence
• Unable to know what is important at the moment, particularly with real-world situations
• Assumes that each piece of evidence is worth the same weight
• Is less useful in intelligence applications when probabilities are initially determined to be 50/50

How-To:

1. Start with a rudimentary estimation for the likelihood of an event occurring
2. Take evidence and apply/use probabilities for potential outcomes
3. Follow the formula whereby P is the probability, E is the event of interest, H is the factor likelihood, and E|H is the probability of the event of interest given the factor occurred in conjunction to it.
4. Do the math to find the probability percentage.
5. This process can be repeated as additional information is given/discovered in order to improve the probability estimate.

Personal Application of Technique:

The class was tasked with finding out the likelihood that an individual, Bob, drove to work based on the fact that he was late. Bob can choose between three different ways to get to work: car, bus, or commuter train. If he takes his car, there is a 50% chance that he will be late. The bus has a 20% chance of making him late while the train has a 1% chance of making him late.

The first question that the class was given was to find out the likelihood that Bob drove to work based on the probability that Bob chose evenly between the three choices. Using Bayesian analysis, the class came up the a 70.4% probability that Bob drove to work that day.

The second question added new information about Bob’s normal transportation habits. Bob’s co-worker knew that Bob almost always takes the commuter train, never takes the the bus, and takes the car 10% of the time. With this new information, the class was able to update the probability that Bob drove to work to 84.75%.

This exercise demonstrated that the application of Bayesian Analysis is not always the most direct method to get to the probability of the situation in that with this particular case, the boss could specifically ask Bob how he got to work that morning. As individuals not seeking a degree in mathematics, there was a significant hesitation to apply the formula and present the findings.

Rating:  3.75 out of 5 stars

Note: The analysts feel this methodology has very strong benefits and is widely applicable, however, it is relatively weak in terms of its application to the intelligence community, particularly in the reliance on numerous factors which affect the utility.  

For Further Information:

Hubbard, D. W. (2010). How to measure anything: Second edition. Hoboken, NJ: John Wiley & Sons, Inc.

Summary of Findings (White Team): Bayesian Theory ( 4.5 out of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the 8 articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in April 2013 regarding Bayesian Probability Theory specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:
According to Hubbard, Bayesian Theory is a relationship of probabilities and “conditional” probability (Hubbard, 2010). Bayesian Analysis allows for the addition of new information as it is learned to decrease uncertainty in the hypothesis. This method allows a quantitative value to be applied to intelligence questions (ISBA, 2009). Additionally, it is very effective in reducing uncertainty.

Strengths:

1. Applicable to the intelligence field because it gives a probability of likelihood rather than statistical significance, such as in frequentist statistics.
2. Allows for updates to probabilities with addition of new information.
3. Allows you to move out of the confirmation bias mindset by asking questions such as what would I observe if X were true or what would I expect to see if X were true?
4. Effective when odds are very high or very low.
5. Is more intuitive than traditional frequentist statistics.

Weaknesses:

1. Less helpful when initial probability is close to 50/50.
2. A piece of evidence that may may not be accounted for may turn out to be the most important.
3. It is difficult to decide how my weight to put into each new piece of evidence.
4. Sometimes difficult to see the real-world application and relevance.
5. Possibly time consuming to conduct bayesian theory to situations.
6. Can sometimes be fairly complex and difficult to understand.

Step by Step Action:

1. Find a topic for which you want to estimate a probability.
2. Assign probabilities to each hypothesis.
3. Update the equation by adding new hypothesis when new evidence is available.
4. Calculate the likelihood a certain scenario will occur based on the probabilities of each hypothesis.
5. For the numerator, multiply the hypothesis being tested by the probability that it will occur.
6. For the denominator, multiply each individual hypothesis by the probability of that event occurring with that hypothesis and find the sum of each probability.
7. Divide the numerator by the denominator to find the final probability.

Exercise:
In class we conducted a simple Bayesian Theory problem to determine the likelihood that an employee would show up to work late by car. The formula that was used in this Bayesian approach was the general form of bayes theorem equation as shown below. There were two parts to the exercise, part A included the bosses probabilities that Bob would come late to work by car, bus, or train, overall determining the probability that Bob came to work late by car. Part B analyzed the same situation, but the probabilities of how a co-worker thought Bob would come late to work by car. By calculating both Part A and Part B it was noticed how the probabilities of Bob coming to work late by car changed with different evidence placed into the general form bayesian equation. In terms of real world applicability, the exercise demonstrated how applying bayesian theory to real life scenarios may be difficult to initiate or not seem as the best approach to figuring out a situation. It would seem just asking Bob what mode of transportation he came to work late in would be a much easier approach. However, the exercise does demonstrate the benefits of how bayesian theory can reduce the uncertainty of a question when more evidence is added, which is essential in the work conducted by intelligence analysts. Overall, Bayesian Theory can provide a much more reliable estimate for the analyst.

Is It Safe To Go Out Yet? Statistical Inference in a Zombie Outbreak Model

Summary: 
The authors Calderhead, Girolami, and Higham (2010) wrote a paper dealing with the potential outcomes of a zombie outbreak using Bayesian theorem to support their conclusions.  Since there has never been a zombie outbreak it is logical to use Bayesian theory to account for unknown data that can be estimated.  Estimations can be made regarding a zombie outbreak, and in turn these estimations can be used in particular ways to end with likely outcomes.

The idea for using Bayesian theory applied to zombie outbreaks starts with a logical probability.  In the case of this paper, the authors state that in one day the far extremes of probability are that no human will turn into a zombie and all humans are converted into zombies.  This probability (labeled prior) is then updated as new data is used (such as different quantity of days) and thus this posterior distribution then becomes the prior and the process is repeated.

The authors then state that many questions can be answered by successfully finding a likely distribution for human to zombie conversion rates.  Such questions include how many soldiers should be mobilized, the scale of quarantine needed, and whether or not it is alright to leave a hiding spot given the number of zombie sightings during a particular time span.  The authors also emphasize that since the rate of change from human to zombie is likely to not be constant, the beta (conversion coefficient) should be a range and not a singular number.

One of the model comparisons the authors use is the comparison of two models: one that assumes zombies can attack alone while the other shows that after a rumor is circulated, zombies are believed to only travel in pairs.  The authors then seek to disprove the second model through Bayes factors (posterior odds = Bayes factor * prior odds), in which statistical evidence for the first model is weighed heavier than the second.  The authors find that the first model with the least amount of noise (introduced as Gaussian distributed noise) is more likely.  This means that the experimental data deviated the least from the expected curve.  Adding additional noise negatively impacted the Bayes factor (shows as how strong the evidence was against the second model.

Another example the authors used Bayesian theory on was for answering the question of whether or not it is safer to leave a hiding spot based on the number of zombies spotted in the past few days.  The authors use two types of analysis for this: one that does not have any observations of previous day's zombie sightings and one that has five day totals of zombie sightings.  The stated the zombie sightings for the second case were 123, 127, 104, 92, and 74.  The left column of the figure below shows Bayesian factors applied to the first model mentioned above, and the potential outcomes.  The right column shows this process with a second layer of data (the five days of observations) which greatly reduces the uncertainty regarding potential zombie totals for the next 45 days.  Thus, due to these observations and using Bayesian theory uncertainty can be greatly reduced and the chance of surviving longer during a zombie outbreak are much higher.

Critique:
I found this article to be fairly complex to read with no experience with Bayesian theory.  However, I really appreciate the application of Bayesian theory to a zombie outbreak.  Although this article topic is (most likely) fantastical, it is well constructed and thoughtful.  Honestly, the topic caught my eye and I doubt I would have tried as hard as I did to understand Bayesian theory had it been on a drier topic.

One issue I had with this article is that it was clearly meant for someone that had previous experience with Bayesian theory.  At times the authors referenced various aspects of Bayesian theory without defining them.  For example, the authors did not just list Bayes factors but instead referenced an article.  For the average reader this does not make reading and understanding this topic any easier.  Additionally, simple Bayesian theory is not that long or difficult to write out and would have saved me time having to look it up to double check I was thinking about the correct thing.

This article is not related to intelligence, save for if there were to ever be a zombie attack it would prove useful for intelligence analysts to extend their lifespans.  However, this model could be applied to medicine for spread of infectious diseases if the transmission rate is unknown.  Instead of zombies and humans there would be infected and healthy individuals.

Source:
Calderhead, B., Girolami, M., & Higham, D. (2010). Is It Safe To Go Out Yet? Statistical Inferencein a Zombie Outbreak Model.University of Strathclyde, United Kingdom.  Retrieved from http://www.strath.ac.uk/media/departments/mathematics/researchreports/2010/6zombierep.pdf