Summary of Findings:  Decision Tree

(Intelligence Application: 2.5 Of 5 Stars)

(Operational Application: 4 Of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in September 2017 regarding Decision Tree as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Decision tree analysis is a flexible method of quantifying possible decisions via branching values based off of simple arithmetic. It is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. Although it is more commonly used as an operational method, it can be used to estimate possible decisions of rival organizations.

Strengths:

• Provides a clear visual representation of all possible decisions
• Cost-effective for the company or organization
• Easy to implement post-data verification
• Easy to communicate it to the decision maker
• Can be used to determine probabilities of different options or outcomes
• Can be used for both quantitative and qualitative data
• It can be combined with other analytical technique
• Can be easily modified if new alternatives are found

Weaknesses:

• Decision trees are extremely data dependent. Without reliable data, final estimates can be skewed
• Tends to be more of an operational method
• Becomes increasingly more complex as more decisions are implemented
• Allows for mirror imaging bias if complete data is not provided
• Expectation vs. reality: outcomes based off of actual decisions made may not be the same as predicted outcomes
• Assumes the creator of the decision tree knows all possible variables

How-To:

1. Decide on a question you are trying to answer
2. Define the problem or decision that is in question
3. Start with a box or square that contains the decision you are trying to answer
4. Gather relevant background data along with all desired or possible outcomes
5. Begin to draw lines or branches out from your original question for each of the decisions you came up with, one branch for each possible choice
6. Associate the costs of implementation of each decision
7. Add the possible outcomes of these decisions and then assign probabilities (the probabilities must equal 100% to account for all possible outcomes)
8. Assigned projected gains or losses to the projected outcomes
9. Multiply the probability of each outcome with the outcomes projected gain or loss
10. Add these values together for each outcome associated with that particular decision
11. Choose decision with optimal outcomes based on equations

Application of Technique:

To demonstrate the application of the decision tree methodology, the class was led through two individual trees to understand the mechanisms. The first decision tree was a simplistic version of whether to invest in a candy cart or a lemonade stand. The class learned the fundamentals of a decision tree by examining the different components and their interactions. Generic values were substituted in for the example to emphasize the process of how to calculate the expected desired outcome.

Following the basic introduction to decision trees, the class undertook a more complex tree and calculated the expected desired outcome of a marketing company. The possible decisions included cutting costs, increased advertising, or to do nothing. Two new concepts: null decision and cost implementation were introduced in the second tree. Null decision would be selected if all other results came back negative. Cost implementation is the cost subtracted from the income shown for each decision.

For Further Information:

1. Decision Tree Basics
2. Decision Tree - Wiki
3. Comparative Study of Decision Tree Analysis vs Similar Methodologies

Decision Trees for Forecasting

Summary and Critique by Matthew Haines

 Summary:

Jakob Uvila discusses the use of decision trees as a forecasting model, and the evolution of traditional decision trees in business. Uvila starts by outlining the history of decision trees and how they have been in use since the 1950s. He also outlines that the reason decision trees are so useful is that the costs associated with them are low. This allows a company the ability to, at minimum, do a decision tree analysis no matter the costs allocated to strategic planning. Then Uvila begins to delve into his comparative case studies with the goal of showing a decision tree’s ability to combine assessments of judgement with data.

Uvila begins to describe the decision tree analysis that was developed by Honeywell Inc. This analysis is used in evaluating which products the company should invest in based on their projected success in the market compared to the option of doing nothing. Uvila is careful to stress that this process is done in two steps. First a decision tree on each individual prospective product, and then a decision tree the combines those trees accounting for interaction of those prospective products. This second step advances the company’s ability to meet future sales goals based on product projection. Uvila stated that the decision tree analysis revealed that the reason there were oddities in the statistical projection of sales in 1988 was the uncertainty of how many products would be in full production in 1988.

Uvila then highlights some of the problems that decision trees deal with and an example of these challenges in a separate case study. He states that the three problems are:

1. The decision maker must have only the information modelled to the left of the subsequent act and will choose among only those actions specified.

 2. The model following the subsequent act must be identical to that which would be specified at the time of the later decision.

3. The decision maker will choose to maximize expected value at the time of the subsequent act

To showcase these challenges, Uvila outlines the AIL Division of Culter-Hammer inc’s opportunity to buy a patent. Specifically, he examines the fact that if the patent was purchased a massive company effort would need to be launched into investigating the patent itself. To assume this information would be erroneous in the decision tree so Uvila states that

an assessment was made of the chances that different subsequent actions would be taken. This assessment took into account whatever relevant information might be known at the time. The mechanism of the analysis then proceeded as usual, treating the actas-event node as any other event node

Critique:

            This article is a great tool to define how a decision tree analysis should be done and its limitations. It does not, however, offer much information as to how effective a forecasting tool it is. It would be good to see a more in-depth evaluation of the decision tree analysis, but the article does a good job of highlighting the strengths of the method. Specifically, that the method can be complex and sound in statistical projections, but it can also be simplified to give decision makers an easily explained visualization. This versatility alone makes the method valuable to the analyst.

Source: Ulvila, J. W. (1985). Decision Trees for Forecasting. Journal Of Forecasting, 4(4), 377-385. http://eds.a.ebscohost.com/eds/pdfviewer/pdfviewer?vid=4&sid=c0bd4b08-3e05-4bd9-b59c-b78e287f53fe%40sessionmgr4007

A Decision Tree Method for Finding and Classifying Names in Japanese Texts

Summary and Critique by: Jared Leets

Summary:

Sekine, Grishman, and Shinnou state that the purpose of the study is to present a learning system that will find and classify name entities in Japanese newspapers. The task was to find named entities and character types such as the name of an organization, date, and location to name a few. The algorithm for the system has several processes, the first step creates the decision tree from the training data while the other is used for making the tagged output from the decision tree. They use a Japanese morphological analyzer (JUMAN) and a program package for the decision tree. The three feature sets in the decision tree include: the part of speech tagged by JUMAN, character type of information, and special dictionaries which were based on JUMAN dictionary entries, entry lists on the internet, or based on human information.

Training sentences are broken up and parts-of-speech are tagged by JUMAN. Then a token is analyzed by a character type and eventually matched against entries in the special dictionaries. A single token has the ability to be matched in entries in many dictionaries. A decision tree can be built from the training data. It can learn from the opening and closing of named entities on the three kinds of information, those being the parts of speech, character type, and special dictionary. To find probabilities of the opening and closing of a named entity for every single token, the properties of all the tokens are analyzed against the decision tree. Once the tokens are assigned in a sentence, the next step is to discover the most consistent likely path through the sentence. The article concludes by stating how they used a decision-tree system to discover and classify names in Japanese texts.

Critique:

The study using decision trees proved to be successful. Their system did not require additional methods and multiple possibilities could be resolved by the probabilistic method. Decision trees seem to be more domain independent compared to the dictionaries in the study. Since their study was smaller compared to other similar studies done with decision tree methods, it had less errors and seemed to be much more positive with the end results. The study was a good example of using the decision tree methodology and helped show how algorithms with the methodology can classify information and show the results. Other studies using decision tree methodology will likely continue to display trends and regularities.

Source:

Sekine, S., Grishman, R., & Shinnou, H. (1998, August). A decision tree method for finding and classifying names in Japanese texts. Proceedings of the Sixth Workshop on Very Large Corpora.http://www.cs.nyu.edu/~sekine/papers/wvlc98.pdf

Decision tree learning used for the classification of student archetypes in online courses

By 

Alexandru Topırceanu

Department of Computer and Information Technology, Politehnica University Timisoara, Bd. V. Parvan 2, 300223 Timisoara, Romania

& 

Gabriela Grosseck

Faculty of Psychology and Sociology, West University Timisoara, Bd. V. Parvan 4, 300223 Timisoara, Romania

21st International Conference on Knowledge Based and Intelligent Information and Engineering Systems, KES2017, 6-8 September 2017, Marseilles, France

Summary:

This paper looks into the student profiles using decision tree and supervised learning. The researcher uses the responses from an online questionnaire to gather detailed opinion from 632 students from Romania regarding the advantages and disadvantages of Massive Open Online Courses(MOOCs), as well as the reasons for not joining online courses. Based on the extracted statistics, they present six decision trees for classifying the finalization and participation rates of online courses based on the students’ individual traits.

To support this direction of research in educational science, they rely on decision tree learning techniques to go beyond the simple statistical analysis and profiling of students done in eLearning. To improve the efficiency of eLearning, the goal of this study is set to define a set of archetypes which can quickly assess any student, so that educators better understand their inner drives to participate and finalize a course.

Decision tree learning is used into link observations about entities (represented in the tree’s branches) to conclude upon the entities’ target value (represented in the tree’s leaf nodes). It is consistently used in data mining and represents a predictive modelling approach. Based on what values the target variable can take, there are classification trees, where the target variable can take a finite set of values, respectively regression trees, where the target variable can take continuous values; for this study they only make use of classification trees

The paper classifies the result with six decision tree as below

1. Finalization rate based on perceived advantages of online course.

2. Finalization rate based on ongoing course evaluation.

3. Participation rate in online courses based on demographics.

4. Course completion based on perceived disadvantages of online courses.

5. Desire for free course based on the reason for not participating in online courses.

6. Desire for certification based on reasons for not participating in online courses.

The paper describes each tree by defining student profiles for each of the unique branches in the tree. Thus, one or more leaf nodes represent a student profile, and the branches that lead to those nodes represent the characteristics of each profile.

Critique: 

This is a well-researched article which uses various techniques in combination with decision tree to research advantages and disadvantages of Massive Open Online Courses(MOOCs). This article publishes final result with much descriptive explanation on how they have used the Decision tree. The results are based on online questionnaire by 632 students, which is smaller a sample and all the participants were from only Romania. When this method is properly implanted, the results can be very useful in enhancing the outcome of courses based on the extracted knowledge from this study. It will also provide decision makers with a better understanding of students’ enrollment pattern to courses and to provide suggestions for structuring course offerings. 

Reference: https://tinyurl.com/ydzexpqf

Simplifying Decision Trees

Simplifying Decision Trees

Summary and Critique by Michael Pouch

Summary:

J.R. Quinlan offers different techniques to help simplify the making of decision trees while at the same time maintaining their accuracy. The purpose of the study is to simplify decision trees by using four methods that help prove their fluency and the ability to apply the knowledge.

The author begins to explain that in order to build knowledge based systems, the ability to function at an expert level in some task does not necessarily confer a corresponding ability to articulate this know-how. Therefore, all this knowledge that the “knowledge engineer” produced, tends to state the engineer’s conclusions and reasoning in broad terms for the analysis. Thus, Quinlan examines four methods to help simplify the method going into creating decision trees. On the whole, three of the methods that Quinlan examines involve pruning the decision tree by replacing one or two subtrees with leaves. The fourth method that Quinlan examines reformulates the decision trees as a set of production rules.

The first method that Quinlan introduces is the method of Cost-Complexity Pruning.  After creating a large decision tree, we look at the subtrees that have one to zero nodes. To prune a tree, we look at these nodes that can be evaluated based on its classification accuracy with the other nodes. Next, we denote these nodes that are to be pruned because they are not focused on the overall validation of the tree. The basic idea of cost-complexity pruning is not to consider all pruned subtrees, but only those that are the “best of their kind” in a sense to be defined.

The second method introduced is Reduced Error Pruning. First, you prune the nodes that are classified as the subtrees and you remove them to make one leaf. Nodes are removed on the basis of choosing the node whose removal most increases the decision tree accuracy on the graph. Pruning process continues until further pruning is harmful.

The third method is called Pessimistic Pruning, where it’s based on a single pass algorithm on a True or False approach. At each step, we remove one rule, such that its removal brings the lowest valued node among all possible removals of the rule in the current tree.

The fourth method simplifies the production rules that go into decision trees. As decision trees classify a case and the leaves are satisfied along the path of conditions, examining the conditions or rules along the path could be generalized on the left side of the tree. Once generalize, there is a certainty factor that comes along the cases. To classify a case, you must find a rule that applies to it. If there is more than one, choose the rule with the higher certainty factor. Overall, this method is a condition-elimination strategy.

Overall, J.R. Quinlan intention is to suggest methods for simplifying decision trees without compromising their accuracy.

Critique:

Overall, I find the first three methods easy to interpret and visualized. The pruning methods are helpful due to their ability to narrow and scope the tree the way that fits the overall validation. When the goal is to produce a sufficiently accurate compact concept description, pruning is highly useful. However, there is a lot swapping of conditions throughout the process. In reaction, decision trees perform well if a few highly relevant attributes exist, but less so if many complex interactions are present.  Since most decision trees divide the leaf into mutually exclusive regions to represent a concept, in some cases the tree should contain several duplications of the same sub-tree to represent the classifier. To conclude, decision trees provide a framework to consider the probability and payoffs of decisions to help the analyst evaluate the likely course, however, it is impossible to plan for all contingencies when it comes to its outcomes. Though the process is simplified, there is no statistical backing if it actually works.

Reference:

Quinlan, J. (n.d.). Simplifying Decision Trees. Massachusetts Institute Of Technology Artificial   Laboratory. Retrieved from https://dspace.mit.edu/bitstream/handle/1721.1/6453/AIM- 930.pdf?sequence=2.

A Decision Tree Method for Building Energy Demand Modeling

Summary and Critique by Claude Bingham

A Decision Tree Method for Building Energy Demand Modeling 

Summary 

Energy consumption has been identified as a major factor in long-term building impact. Additionally, newer buildings have consumed more and more energy over time. To that end, the researchers in this project wanted to construct an accurate predictive model that would be able to estimate future energy use of buildings.

They chose decision tree methodology to create the predictive models. Regression methods were noted to be too complex for users with limited mathematical training; the researchers saw neural networks as 'black boxes,' unreplicable for some of the same reasons of regression. Building simulations cannot accurately predict building occupant behavior patterns and therefore can only estimate what a building's energy consumption could be in a statically situational environment. Decision trees, however, is relatively simple, can manipulate numerical and categorical data, and does not require much computation.

Decision trees use a flowchart-like structure to show hierarchy, status, and category of data. In this study, for example, a decision tree depicted the outside temperature, if a room was occupied, and whether the air conditioning was on because of those previous two factors. Based on the number of recorded occurrences of each possible variable state, energy use can be approximated for an individual room.

To verify the actual ability of such a model to create reliably accurate predictions, the research team used the C4.5 decision tree algorithm with open-source WEKA data-mining software. This pairing was chosen for their flexibility and ability to apply multiple types of data. The constructed model is then tested against predicted values. In this research study, the model was constructed to include six categorical variables and four numerical variables based on data collected from 80 buildings in six districts in Japan. The resulting value was set to be either 'HIGH' or 'LOW' energy use intensity.

The test model was able to correctly predict 92% of expected cases. The researchers noted that the confidence interval was 80%, too low to be consistently reliable and the model was miss-attributing variables at times. This was likely due to the size of the data set and limited variable hierarchy (also tied to data set size and variety).

Critique

This research benefited greatly from examining reasons for and against using various methodologies for predictive studies. The experiment was well-explained and well executed, with one exception. The sample size for the test data was too small. While the results were both reasonably accurate, and passably reliable, it shows decision trees are not downward scalable for smaller data samples. This methodology appears to work well with large data sets, but not smaller ones.

Predicting Recovery in Patients suffering from Traumatic Brain Injury by using admission variables and physiological data

Predicting Recovery in Patients Suffering from Traumatic Brain Injury By Using Admission Variables and Physiological Data: A Comparison between Decision Tree Analysis and Logistic Regression

Authors: Peter J.D. Andrews, M.D., F.R.C.A., Derek H. Sleeman, Ph.D., F.R.S. (ED), Patrick F.X. Statham F.R.C.S (Sn), Andrew McQuatt M. Sc., Vincent Corruble Ph.D., Patricia A. Jones, M. App. Sci., Timothy P. Howells, Ph.D., and Carol S. A. Macmillan, M.R.C.P., F.R.C.A

Summary:

This study compared results given by logistic regression and decision tree analysis on patients suffering from Traumatic Brain Injury. The researchers studied 124 patients in an intensive care unit over a 12 month period by using a data collection system. They input the values into a logistic regression, then created a decision tree from root nodes to target classes. This is called the Glasgow Outcome Score.

The researchers then assessed the outcome after 12 months of 69 patients with 8 insult or head injury categories. They input the data into a logistic regression to assess how patient age, Glasgow Coma Scale Score, Injury Severity Score, pupilary response upon admission, and insult duration. After they input the data, they found hypotensive, pyrexic, and hypoxemic insults to be the most significant predictors of mortality. 

After they used decision tree analysis, they found that hypotension and low cerebral perfusion pressure to be the most significant predictors of death. The analysis proved to 9.2 percent more accurate than just using the largest outcome category as a predictor of mortality.

Critique:

The article gave a good explanation for why decision tree makes for more accurate analysis. It also did a decent job of listing the variables used for the analysis. However, this is one study done on the decision tree analysis in the medical field and more specifically dealing with brain injuries. The study will need to be replicated using different numbers of patients to give an complete assessment of how accurate decision tree analysis is in that field.

Citation:

http://thejns.org/doi/pdf/10.3171/jns.2002.97.2.0326. 

Summary of Findings: Gap Analysis (2 Of 5 Stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in September 2017 regarding Gap Analysis as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Gap analysis is a methodology that allows one to estimate the “gap” in between the current situation of a company and a specific goal of the company. Operational gaps tend to look internally, while intelligence gap analysis looks at the external environment and what their competitors’ strategies are. This highlights a fundamental difference in what is meant by gaps, where intelligence gaps are essentially missing pieces of a puzzle, while operational gaps are related to obstacles to be overcome.  

Strengths:

• It helps to prepare preliminary list of gaps per scenario.
• Helps identify what areas to improve upon
• Essentially is an internal audit of the organization
• Gives a picture of “where you are” and “where you want to be” to provide estimates on how to close the gaps
• Help identify an objective and how to plan a course action

Weaknesses:

• Little to no evidence gap analysis is an effective methodology
• Gap analysis might generate misleading results, if the individuals carrying out the analysis do not conduct a proper trend analysis beforehand.
• Gap analysis doesn’t address complexities when analysis gaps
• Can lead to more operational based questions if not appropriately used
• Ignores the possibilities of biases if used incorrectly
• Typically will not be effective to use if company does not have the necessary resources
• 
  

How-To:

1. Identify the strategic goal.
2. Select most appropriate measure of potential gap (qualitative/quantitative).
1. Make a list of triggers to which led to gaps
3. Collect information about current state.
4. Analyze difference between current state and goal.
1. Compare situational difference between current state and strategic goal (qualitative).
2. Calculate quantitative difference between current state and strategic goal (quantitative).
3. Look at external environment in which the company is operating i.e. industry, competitors.
5. Discuss what further topics and questions need to be answered in order to reach desired goals.
6. Assign action for all gaps

Application of Technique:

To demonstrate the application of the Gap Analysis methodology, the class looked at an example case study depicting the gap in service quality between T-Mobile and AT&T. The class was instructed to research service coverage, plan prices, customer service, and product quality and compare qualitative or quantitative differences between the two service providers in those areas. Multiple areas showed both qualitative and quantitative areas with gaps. While going through the process, the analysis subtly changed from an intelligence exercise into theoretical operational brainstorming. Overall, while the analysis answered the initial question of what gaps existed between T-Mobile and AT&T, the methodology did not prove to be a valid way to increase analysis quality.

For Further Information:

1. Towards Total Product Quality Gap Analysis
2. Gap Analysis - Wikipedia
3. Gap Analysis Definition - Search CIO

GAP ANALYSIS REVISITED

Summary:

Gap analysis is a methodology, which is often used by various organizations and individuals seeking to reduce overall inefficiency or gaps in their respective environments. These gaps or inefficiencies may vary in their nature, scope and complexity, but the same analytical process can be applied to resolve them, if one chooses to use gap analysis. According to the article, Gap Analysis revisited, written by Chevalier in relation to the “International Society for Performance Improvement’s 10 Standards of Performance Technology”, the methodology consists primarily in establishing the existing and desired performance levels, then working to bridge the gap between the two by setting reasonable goals that will help achieve the desired outcome. According to the author, determining the existing and desired levels of performance is the starting point while conducting gap analysis. Nonetheless, the author stresses the fact that this it is primordial that this process be followed by the chartering of plausible targets for bridging the said gap. These “reasonable targets” he argues, must be broken down in smaller objectives and laid out in relevant and comprehensive terms to ensure that every participant involved in the exercise feels a sense of ownership, and thus is motivated and able to work towards achieving these targets. In the article, Chevalier provides us with an example that relates to this notion. In this example he mentions that the Netherlands swimming team decided to increase its performance level in the 100-meter free-style event from 51 seconds to near the world record of 47.84 seconds. To achieve this challenging but attainable goal, the team decided to reduce this gap of 3 seconds into smaller intermissions of .02 second a week and .004 second per training day respectively. This way the team was able to reduce the difference in their existing performance level of 51 seconds to 48 seconds which was their desired level of performance, by breaking down their overall target into smaller reachable objectives.

Ultimately, though it is evident that reducing the void between the existing and desired levels of performance and setting goals to that end is the essence of gap analysis, the author of the article argues that without well informed trend analysis, the outcome of conducting gap analysis might prove to be erroneous. To support this argument, Chevalier contends that depending on the nature of the existing performance level, the outcome of the gap analysis will either be misleading or correct. For example, he states that in a given organization, if there was an upward trend before bridging the gap, then continued upward performance may not necessarily be an indication that that the actions to increase efficiency added any value.

Critique:

Chevalier does a great job in demonstrating how one can apply gap analysis in order to improve performance levels in various fields. Nonetheless, there seems to be a major flow in his study of the analytical technique. Although it is clear that identifying “where you are” and “where you want to be” and setting objective to that end is the way to go, there seems to be a missing a link. It is great to carry out these steps, but how efficient are they if one has not determined the underlying problem which created the gap in the first place. For example, while trying to reduce their time in the Olympics from 51 to 48 seconds, did the Netherlands swimming team identify why they were below average to begin with? Therefore, the article should have included the process of discerning underlying problems in performance levels before targets are set to increase efficiency.

Source:

http://eds.a.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=14&sid=d05a93f9-a98c-445e-9eac-44ad0b5f9b1f%40sessionmgr4010