An Analysis of Multi-Criteria Decision Making Methods

Mark Velasquez and Patrick T. Hester (2013)

Summary

This paper performs a literature review of common Multi-Criteria Decision Making methods, examines the advantages and disadvantages of the identified methods, and explains how their common applications relate to their relative strengths and weaknesses.

Multi-Attribute Utility Theory (MAUT)

MAUT is an expected utility theory that can decide the best course of action in a given problem by assigning a utility to every possible consequence and calculating the best possible utility. The major advantage of MAUT is that it takes uncertainty into account. It is comprehensive and can account for and incorporate the preferences of each consequence at every step of the method. This amount of accuracy is convenient, however it can lead to many possible disadvantages. An incredible amount of input is necessary at every step of the procedure in order to accurately record the decision maker’s preferences, making this method extremely data intensive. The preferences of the decision makers also need to be precise, giving specific weights to each of the consequences, which requires stronger assumptions at each level. MAUT has seen heavy application in economic, financial, actuarial, water management, energy management, and agricultural problems.

Analytic Hierarchy Process (AHP)

AHP is a theory of measurement through pairwise comparisons and relies on the judgments of experts to derive priority scales. One of its advantages is its ease of use. Its use of pairwise comparisons can allow decision makers to weight coefficients and compare alternatives with relative ease. It is scalable, and can easily adjust in size to accommodate decision making problems due to its hierarchical structure. Besides, it is not nearly as data intensive as MAUT. Due to the approach of pairwise comparisons, it can also be subject to inconsistencies in judgment and ranking criteria. One of its biggest criticisms is that the general form of AHP is susceptible to rank reversal. Due to the nature of comparisons for rankings, the addition of alternatives at the end of the process could cause the final rankings to flip or reverse. AHP has seen much use in performance-type problems, resource management, corporate policy and strategy, public policy, political strategy, and planning

Fuzzy Theory

Fuzzy set theory is an extension of classical set theory that allows solving a lot of problems related to dealing the imprecise and uncertain data. It has many advantages. Fuzzy logic “takes into account the insufficient information and the evolution of available knowledge. It allows imprecise input. It allows a few rules to encompass problems with great complexity. For disadvantages, fuzzy systems can sometimes be difficult to develop. In many cases, they can require numerous simulations before being able to be used in the real world. Fuzzy set theory is established and has been used in applications such as engineering, economic, environmental, social, medical, and management.

Case-Based Reasoning (CBR)

CBR is a MCDM method that retrieves cases similar to a problem from an existing database of cases, and proposes a solution to a decision-making problem based on the most similar cases. This provides the first of its advantages, which is that it requires little effort in terms of acquiring additional data. It also requires little maintenance as the database will already be existing and requires little upkeep. One major advantage that it has over most MCDM methods is that it can improve over time, especially as more cases are added to the database. It can also adapt to changes in environment with its database of cases. Its major drawback is its sensitivity to inconsistency in data. Previous cases could be invalid or special cases may results in invalid answers. Sometimes similar cases may not always be the most accurate in terms of solving the problem at hand. CBR is used in industries where a substantial number of previous cases already exist. This includes comparisons of businesses, vehicle insurance, medicine, and engineering designs.

Data Envelopment Analysis (DEA)

DEA uses a linear programming technique to measure the relative efficiencies of alternatives. It rates the efficiencies of alternatives against each other, with the most efficient alternative having a rating of 1.0, with all other alternatives being a fraction of 1.0. It has a number of advantages. It is capable of handling multiple inputs and outputs. Efficiency can be analyzed and quantified. It can uncover relationships that may be in hidden with other methods. An important disadvantage is that does not deal with imprecise data and assumes that all input and output data are exactly known. In real world situations, however, this assumption may not always be true. The results can be sensitive depending on the inputs and outputs. DEA is used wherever efficiencies need to be compared. This is commonly used in economic, medical, utilities, road safety, agriculture, retail, and business problems.

Other Methods

SMART: SMART is one of the simplest forms of MAUT. It requires two assumptions, namely utility independence and preferential independence. This method conveniently converts importance weights into actual numbers. Major advantages of SMART, in addition to those described in MAUT, are that it is simple to use and it actually allows for any type of weight assignment techniques. It requires less effort by decision makers than MAUT. It also handles data well under each criterion. Like MAUT, a disadvantage is that the procedure for determining work is not convenient considering the complicated framework.

Goal Programming: Goal Programming is a pragmatic programming method that is able to choose from an infinite number of alternatives. One of its advantages is that it has the capacity to handle large-scale problems. Its ability to produce infinite alternatives provides a significant advantage over some methods, depending on the situation. A major disadvantage is its inability to weight coefficients. Many applications find it necessary to use other methods, such as AHP, to properly weight the coefficients. Goal programming has seen applications in production planning, scheduling, health care, portfolio selection, distribution system design, energy planning, water reservoir management, timber harvest scheduling, and wildlife management problems.

ELECTRE: ELECTRE, along with its many iterations, is an outranking method based on concordance analysis. Its major advantage is that it takes into account uncertainty and vagueness. One disadvantage is that its process and outcomes can be hard to explain. Further, due to the way preferences are incorporated, the lowest performances under certain criteria are not displayed. The outranking method causes the strengths and weaknesses of the alternatives to not be directly identified, nor results and impacts to be verified. ELECTRE has been used in energy, economics, environmental, water management, and transportation problems.

PROMETHEE: PROMETHEE is similar to ELECTRE in that it also has several iterations and is also an outranking method. Its advantage is that it is easy to use. It does not require the assumption that the criteria are proportionate. The disadvantages are that it does not provide a clear method by which to assign weights and it requires the assignment of values but does not provide a clear method by which to assign those values. PROMETHEE has seen much use in environmental management, hydrology and water management, business and financial management, chemistry, logistics and transportation, manufacturing and assembly, energy management, and agriculture

SAW: SAW is a value function is established based on a simple addition of scores that represent the goal achievement under each criterion, multiplied by the particular weights. It has the ability to compensate among criteria. It is also intuitive to decision makers. The calculation is simple and can be performed without the help of complex computer programs. It has specific disadvantages: All the values of the criteria should be maximizing. Minimizing criteria should be transformed to maximizing ones before being used in the analysis. All the values of the criteria should be positive and the estimates yielded by SAW do not always reflect the real situation. SAW has had applications in water management, business, and financial management.

TOPSIS: TOPSIS is an approach to identify an alternative which is closest to the ideal solution and farthest to the negative ideal solution in a multi-dimensional computing space. It has numerous advantages. It has a simple process. It is easy to use and programmable. The number of steps remains the same regardless of the number of attributes. A disadvantage is that its use of Euclidean Distance does not consider the correlation of attributes. It is difficult to weight attributes and keep consistency of judgment, especially with additional attributes. TOPSIS has been used in supply chain management and logistics, design, engineering and manufacturing systems, business and marketing management, environmental management, human resources management, and water resources management.

Critique

This paper assesses the more common methods of MCDM in order to benefit practitioners to choose a method for solving a specific problem. Identification of common MCDM methods and identification of strengths and weaknesses is very useful however; this article solely itself is not enough to determine a method of MCDM. It only gives a general idea and helps researchers during the method choosing process.

Source:

Velasquez, M., & Hester, P. T. (2013). An analysis of multi-criteria decision making methods. International Journal of Operations Research, 10(2), 56-66. Retrieved From: http://www.orstw.org.tw/ijor/vol10no2/ijor_vol10_no2_p56_p66.pdf

Prediction of Future Forest Fires Using the MCDM Method

Saeedeh Eskandari, Jafar Oladi Ghadikolaei, Hamid Jalilvand, and Mohammad Reza Saradjian

Summary:

This article uses a combination of different techniques to assess the risk of forest fires in District Three of Neka-Zalemround Forests in Northern Iran.  The study uses a fire risk model integrated with the MCDM method in a GIS framework to map forest fire risk.  The authors claim that while there have been many fire risk models to incorporate GIS, none have used the MCDM method.  The model developed in the paper will be tested against the historical occurrences of wildfires in order to test its reliability.

The area under examination is in the Mazandaran Province in Iran, and covers an area of 153.07 km².  The minimum altitude is 90 m above sea level, while the maximum is 820 m.  This is an area that has seen numerous wildfires in recent years.

The methodology being utilized consisted of four different components:

• Determination of fire risk major criteria and sub-criteria and preparation of the maps
• Assignment of in-layer and out-layer weights to sub-criteria in a GIS environment
• Combination of sub-criteria and major criteria maps and construction of fire potential map
• Validation of the fire risk potential map

This study is using four major criteria (topographic, biologic, climatic, and human), which are made up of a total of 17 sub-criteria. 

Each sub criteria is modeled as a layer in GIS using raster format.  The in-layer weights of the sub-criteria range from 0 to 1, with 0 being very low and 1 being very high.  The out-layer weights are the four major criteria.  The weights of the sub-criteria add up to be the weight of the major criteria.  In turn all of the major criteria are utilized in calculating the fire risk index.  This is shown below, where w indicated weight.

The maps were developed by using GIS spatial analysis and the raster calculator option in GIS.  The individual cells of the fire risk potential map have been ranked from very low to very high based on these calculations. 

When the fire risk potential map is compared to the map of actual fires the model holds up favorably.  This indicates that the multi-criteria model they used has a good efficiency to predict future fires.  If it were to be used in other areas however, the weights would need to be changed accordingly.

Critique:

This is a well laid out and carefully written article.  It does a great job explaining exactly what the authors did without being overly complicated.  My greatest concern involves the MCDM aspect of the study.  While their results were shown to have a strong correlation with real life fires, this is due to their criteria being properly weighted.  From what was in the article it seems like this would be difficult for someone without both a deep knowledge of the area and a talent for weighting that knowledge to replicate. 

Source:

Eskandari, S., Ghadikolaei, J. O., Jalilvand, H., & Saradjian, M. R. (2015). Prediction of Future Forest Fires Using the MCDM Method. Polish Journal Of Environmental Studies, 24(5), 2309-2314. http://eds.a.ebscohost.com/eds/detail/detail?sid=3623cd0a-2492-425c-885b-fb64d206c48d%40sessionmgr4004&crlhashurl=login.aspx%253fdirect%253dtrue%2526profile%253dehost%2526scope%253dsite%2526authtype%253dcrawler%2526jrnl%253d12301485%2526AN%253d109426727&hid=4113&vid=0&bdata=JnNpdGU9ZWRzLWxpdmU%3d#AN=109426727&db=a9h

Integrating GIS and MCDM Methods

By: Piotr Jankowski

Summary:

In the early 1980s geographical information system (GIS) software emerged commercially as a new information processing technology offering unique capabilities of automating, managing, and analyzing a variety of spatial data. GIS has been depicted as a decision support technology with many applications of GIS developed over the last decade providing information necessary for decision-making in many diverse areas.  One perspective on developing better decision support capabilities of GIS can be identified based on integration of GIS and specialized analytical models.  This perspective improves the decision support capabilities on the expansion of GIS descriptive, prescriptive, and predictive capabilities by integrating GIS software with other general software packages and with specialized analytical models like environmental and socioeconomic models.

This article’s term MCDM is used in reference to multiple attribute decision making (MADM) which is concerned with choice from a moderate/small size set of discrete actions, involving choosing, based on the decision criteria and criteria priorities, from a moderate/small size set of alternatives.

MCDM techniques can be classified according to the level of cognitive processing demanded from the DM and the method of aggregating criterion scores and the DM's priorities.  Two classes of MCDM techniques can be distinguished:  compensatory and non-compensatory.  The compensatory approach is based on the assumption that the high performance of an alternative achieved on one or more criteria can compensate for the weak performance of the same alternative on other criteria.  The compensatory approach is cognitively demanding since it requires the DM to specify criterion priorities expressed as cardinal weights or priority functions.  Under the non-compensatory approach, a low criterion score for an alternative cannot be offset by another criterion's high score. The non-compensatory approach is cognitively  less demanding than  the  compensatory  approach  since  it requires, at the most,  the ordinal  ranking of  criteria  based on  the DM's priorities.  Both the classes of techniques can be further broken down as illustrated in the article. 

DMs can then use five different choice strategies that can be matched with characteristics of different MCDM techniques.  There are:

1.  Screening of absolute rejects: elimination of clearly dominated alternatives as the first step before any further choice deliberation

1. Satisficing principle: the DM will consider all the alternatives  that satisfy conjunctively or disjunctively the minimum  performance levels

1. First-reject: the DM wants to use exclusively the conjunctive elimination rule to reject  all the alternatives that do not pass  minimum  threshold values

1. Stepwise elimination: The DM narrows down the choice re-evaluating  the set of remaining alternatives every time one of  the alternatives is eliminated

1. Generation of linear ordering: the DM wants to generate a ranking of alternatives from the most preferred to the least preferred one

The first four choice strategies can be implemented using exclusively the non-compensatory MCDM techniques with the last strategy requires the full processing approach so it can be implemented using the compensatory MCDM techniques. 

A few of the MCDM techniques can be implemented directly in GIS using the operators of the database query language. Others, however, can be implemented more efficiently using external programs and integrating these programs with GIS. Two strategies for integrating GIS and MCDM are proposed. The first strategy called the loose coupling strategy suggests linking GIS and MCDM techniques using a file exchange mechanism. The second strategy called the tight coupling strategy suggests linking GIS and MCDM techniques using a shared database.

Critique:

Overall, this article was very interesting in how it broken down the various aspects of MCDM into categories, techniques within these categories, choice strategies paired with the techniques, and finally the recommended strategies.  At points, the article was somewhat dense with in depth mathematical explanations for utilizing the various techniques, but was nonetheless very clear in how to perform the various listed techniques of MCDM.   The role of MCDM in GIS is to look for suitable alternatives while helping the DM assign priority weights to decision criteria, evaluate the suitable alternatives, and visualize the results of choice.  While often several results satisfy the minimum threshold values, MCDM techniques are often required to further reduce the alternatives and select the best choice.  Thus, introducing MCDM techniques into the GIS context will likely improve GIS decision support capabilities.  Additionally, the article notes that further research is needed on sensitivity analysis topics in an integrated GIS-MCDM system and facilitating group decision making in the GIS-MCDM context.

Source:

Piotr Jankowski (1995) Integrating geographical information systems and multiple criteria decision-making methods, International Journal of Geographical Information Systems, 9:3, 251-273, DOI: 10.1080/02693799508902036

http://dx.doi.org/10.1080/02693799508902036

Multi-criteria decision making on strategic selection of wind farms

Amy H.I. Lee, Hsing Hung Chen, & He-Yau Kang. (2009).

Summary

In this study, Lee, Chen, and Kang stress the importance of wind farms as a source of renewable energy, but note that deciding where to build a wind farm is a complicated decision that requires many factors, both positive and negative, to be considered. The authors emphasize the importance of utilizing a framework that evaluates projects while considering markets, technologies, social and environmental impacts, etc. The framework they suggest is the analytic hierarchy process (AHP) model with the inclusions of benefits, opportunities, costs, and risks (the BOCR merits).

The authors summarize the ten steps in conducting an AHP model with BOCR.

1. Form a committee of experts in the industry.
2. Construct a control hierarchy for the problem.
3. Determine the priorities of the strategic criteria.
4. Determine the importance of benefits, opportunities, costs, and risks to each strategic criterion.
5. Determine the priorities of the BOCR merits.
6. Decompose the problem into a BOCR hierarchy with four sub-hierarchies.
7. Formulate a questionnaire based on the BOCR hierarchy to pairwise compare elements, or factors, in each level with respect to the same upper level element.
8. Calculate the relative priorities in each sub-hierarchy.
9. Calculate the priorities of alternatives for each merit sub-hierarchy.
10. Calculate the overall priorities of alternatives by synthesizing each alternative under each merit from Step 9 with corresponding normalized weights from Step 5 using one of five methods: additive, probabilistic additive, subtractive, multiplicative priority powers, or multiplicative (for equations please see the article, p. 123).

Lee, Chen, and Kang test this method with a case study of the Chinese wind farm project. The wind farm developers were looking to install 500 wind turbines and needed to determine where to construct the farm. The criteria and sub-criteria for the wind farm project can be seen in Table 1, below.

Table 1: The criteria and sub-criteria for the wind farm project

A questionnaire is designed asking the experts to evaluate the priorities of benefits, opportunities, costs, and risks. Afterwards, the priorities of the alternatives under each merit are calculated. The priority results can be found in Table 2. 

Table 2: Relative priorities of criteria and sub-criteria

The most important benefit criterion was wind availability (criteria (a)), with a score of 0.6317. The important sub-criteria were mean wind power density (a2) and mean wind speed (a3), with scores of 0.2637 and 0.1971, respectively. This means that the greatest benefit for a specific site is having sufficient wind for operations. The most important opportunities were wind power concession program (e1, 0.1945), and clean development mechanisms program (e2, 0.1680), implying that policy support is a major driver to develop wind power. The cost of wind turbines (g, 0.5595) is a major concern, while the risk of concept conflict (j, 0.5639), implies that not all political parties agree on the need for wind power. 

After determining these priorities, the criteria are scored. High scores for benefit and opportunity merits indicate better performance, whereas high scores for cost and risk merits indicate worse performance. The authors evaluated five potential sites, and calculated that one specific site (Site B in the study) was clearly the superior option. All five calculation methods (as previously mentioned in Step 10) agreed that Site B was expected to be the best location for a wind farm, primarily due to the highest wind availability (benefit, the highest overall priority) and is also the second least costly location.

Critique

The authors do a good job explaining how AHP with BOCR works and their application of the method to a real-world case helps clarify the steps. Despite this method being quite math-heavy, the principles that are derived from it can be very useful to an intelligence application of multi-criteria decision making (MCDM), which would be multi-criteria intelligence matrix (MCIM). Specifically, one can draw out the importance of a structured approach, prioritization, and the ability to support the estimate. Additionally, one weakness of MCIM is the necessity to weight or prioritize criteria or alternatives. In this study, the authors distribute questionnaires to a panel of experts and synthesize the results, and in doing so reduce our inability to accurately weight options.

Source

Lee, A.H.I., Chen, H.H., & Kang, H.Y. (2009). Multi-criteria decision making on strategic selection of wind farms. Renewable Energy, 34, 120-126. Retrieved from: Science Direct.

Decision-Making Using Analytic Hierarchy Process (AHP)

Source: http://analytics.ncsu.edu/sesug/2012/SD-04.pdf

Summary:

This paper introduces how to use the Analytic Hierarchy Process (AHP), a famous method that is part of the Multi-Criteria Decision Making process (MCDM). AHP was created to solve issues when facing a mix of qualitative, quantitative, and conflicting factors in the MCDM process. AHP uses the judgements of decision makers to form a decomposition of problems into hierarchies. Each section is further broken down into different levels within the hierarchy which combine with the decision maker’s problem that needs to be solved. The hierarchy is used to derive ratio-scaled measures for alternatives, and the relative value that alternatives have against organizational goals and project risks. AHP contains 4 steps.

Step 1: Define the problem and state the goal or objective.

Step 2: Define the criteria or factors that influence the goal, and structure these factors into levels and sublevels.

Step 3: Use paired comparisons of each factor with respect to each other that forms a comparison matrix with calculated weights, ranked eigenvalues, and consistency measures. (See chart below for comparison matrix example).

Step 4: Synthesize the ranks of alternatives until the final choice is made.

The paper used a family upgrading smart phones to present an example how AHP was applied to make the best choice. 

Three smart phones were considered (1,2,3) and the four qualities that were evaluated were cost, display resolution, battery life and internal storage. 

The first step in the AHP process was to build a table illustrating the problem and comparing each phones attribute. 

The second step involves creating the “parent-child” relationship.

The third step is the most complex part of the process and involves assigning attributes from the matrix shown below step 3. The authors compared each criteria and device choice using a computer algorithm to determine the value of each selection in a measurable format. The graphic below shows the results of the calculations. Without explaining the math behind equations, each family member rated the criteria of what was important to them, and averaged the numbers from the matrix.

The results of the calculations above were further refined using mathematical equations. The chart below displays the results. The most important factor to consider is the Priority column which is the relative ranking of the criteria produced by dividing each element of the matrix with the sum of the column. 

  

Step 4 shows the final rankings of the AHP process based on the Priority and costs benefit ratio to come to a conclusion on the best smart phone to purchase.

Critique:

The authors used SAS/IML statistical software to run the equations which I did not include due to the complexity of the problem. Nonetheless, I attempted to break down the AHP process by reducing as much technical information as possible, yet still highlight how the process works. When applying this process to intelligence analysis, one of the main problems that arises is assigning attributes to the criteria in a matrix. Because many of the issues analyst face are qualitative in nature, analysts may assign a high degree of varying importance to a piece of criteria. Nevertheless, this technique improves forecasting ability only when applied correctly, and can solve problems which have multiple scenarios.

Brainstorming (3.5 out of 5 stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the  articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in October 2015 regarding Brainstorming as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Description:

Brainstorming is a modifier that is used to generate new ideas. There are various types of brainstorming, but they can generally fall under the categories of structured or unstructured. The use of structured brainstorming and the use of unstructured brainstorming can produce slightly different results as the means to produce the results are not the same. This is a very good technique to use in order to get the ideas flowing at the beginning of a project and provide a good starting point.

Strengths:

• Helps to generates new ideas by utilizing and merging different approaches of the participants
• Identifies key drivers in a problem
• Can help generate alternative hypothesizes, outcomes, or scenarios
• When structured, can direct groups toward a topic of focus
• When structured, reduces groupthink and minimizes dominant personalities
• Can incorporate members with diverse backgrounds to broaden range of idea generation
• Can be applied to complex problems

Weaknesses:

• When unstructured, dominating personalities can take over
• When unstructured, can result in groupthink
• When unstructured, can inhibit creative thinking
• Potential for the prevention of idea production or censorship
• The size of the brainstorming group should be determined appropriately
• Structured brainstorming is more time intensive
• Requires consideration toward group makeup
• Potential to lead down a “rabbit hole” if group becomes too narrowed
• No specific set of rules or steps for unstructured brainstorming

How-To:

Nominal group technique was used as a form of brainstorming, and involves a facilitator and members of a team. The facilitator asks an open ended question, and the participants write down their ideas on a piece of paper or 3x5 note card. After the silent brainstorming session, the facilitator writes each member's idea on a white board. The ideas can be separated into categories or voted by each participant. If participants vote for which ideas to pursue, three votes are allotted for each member. The ideas generated give the team a general direction to pursue a topic or tasking the group is responsible for.

Personal Application of Technique:

For the exercise the nominal group technique was used as a form of structured brainstorming. The session begins with a facilitator asking an open ended question; for the exercise the question asked was, “How does ISIS finance their operations?” All the participants are given 5-10 minutes to write their ideas on a piece of paper. After the silent brainstorming session the facilitator asks each participant for one idea in a round robin fashion, and writes down the idea on the whiteboard.  After all ideas are exhausted, the ideas are separated into categories.