CONTENTS

1.0 Criteria Overview

2.0 Assignment Details

2.1 Improving the Objective Assignment

2.2 Transition Paragraph

2.3 Evaluation Criteria and Their Purpose

2.4 Objective Criteria

2.5 Subjective Criteria

2.6 Figures that Support Criteria

2.7 Simple Additive Weighting Method

2.8 Assigning Weights to the Criteria

1.0 CRITERIA OVERVIEW

The Criteria assignment is the third assignment. Before you begin the Criteria assignment, you must have submitted and received instructor feedback on the previous Objective assignment.

The goal of this assignment is to identify, define, and describe an appropriate number (i.e., 3 to 5) evaluation criteria that are important to the decision maker(s). The number of criteria and number of options must add to eight with no less than three of each. The levels of at least three criteria must be objectively measurable (i.e., not subjectively assigned). In this assignment, you do not want to describe any particular option or solution [1]. That will be the focus of the Options assignment.

2.0 ASSIGNMENT DETAILS

This text describes in very prescribed ways how you must complete this assignment. After each prescribed item below, you will see in parentheses the typical number of paragraphs (P) needed (e.g., "1-2P" means "1 to 2 paragraphs are typically needed). The outline also indicates when a figure (F) or table (T) is typically needed. Details about how to create and format figures and tables is in the "Figures and Tables" section of the "Resources" chapter.

Your Criteria assignment must include:

1. An updated and improved version of the Objective assignment.

2. A transition paragraph that helps readers understand the next step of the analysis. (1P)

3. A description of the first criterion, including a short informative name (e.g., "Efficiency" or "Yield"), a rationale for why the decision maker or decision maker group values the criterion, a description of how the author will objectively measure the level of the criterion, and a reasonable minimum (i.e., worst) and maximum (i.e., best) potential level for the criterion. (1-2P and 1F showing example of high and low levels)

4. A description of the second criterion, including a short informative name (e.g., "Efficiency" or "Yield"), a rationale for why the decision maker or decision maker group values the criterion, a description of how the author will objectively measure the level of the criterion, and a reasonable minimum (i.e., worst) and maximum (i.e., best) potential level for the criterion. (1-2P and 1F showing example of high and low levels)

5. A description of the third criterion, including a short informative name (e.g., "Efficiency" or "Yield"), a rationale for why the decision maker or decision maker group values the criterion, a description of how the author will objectively measure the level of the criterion, and a reasonable minimum (i.e., worst) and maximum (i.e., best) potential level for the criterion. (1-2P and 1F showing example of high and low levels)

6. Optional: A description of a fourth criterion, including a short informative name (e.g., "Efficiency" or "Yield"), a rationale for why the decision maker or decision maker group values the criterion, a description of how the author will objectively measure or subjectively estimate the level of the criterion, and a reasonable minimum (i.e., worst) and maximum (i.e., best) potential level for the criterion. Subjectively estimated criteria must use a zero to ten scale where zero is least preferred and ten is most preferred. (1-2P and 1F showing example of high and low levels)

7. Optional: A description of a fifth criterion, including a short informative name (e.g., "Efficiency" or "Yield"), a rationale for why the decision maker or decision maker group values the criterion, a description of how the author will objectively measure or subjectively estimate the level of the criterion, and a reasonable minimum (i.e., worst) and maximum (i.e., best) potential level for the criterion. Subjectively estimated criteria must use a zero to ten scale where zero is least preferred and ten is most preferred. (1-2P and 1F showing example of high and low levels)

8. A description of the simple additive weighting (SAW) method of MCDA and a review of the preference independence of the selected criteria. (2P)

9. A description and rationale for how the decision maker or decision making group weights the criteria. Include a table (i.e., Table 1) that shows vertically each criterion's short name and horizontally each criterion's weight, minimum (i.e., worst) value, maximum (i.e., best) value, measurement approach, and if applicable the unit for measuring the level of each. (1-2P and 1T)

To submit your assignment, follow the assignment submission instructors provided to you by your instructor in the course syllabus or on the course site of your institution's learning management system.

2.1 Improving the Objective Assignment

An important part of this assignment is that you review the instructor feedback on and make improvements to your previous assignment. Keep in mind that, even if you received a favorable grade or positive instructor feedback, you should not interpret that to mean that your instructor believes you do not need to make any improvements. You do! Self-review and peer-review strategies are described in that section of the "Resources" chapter.

One of the most significant ways to improve your work is to add concrete details (e.g., in footnotes) that build your credibility as an analyst. Suggestions for doing this can be found in the "Credibility, Evidence, and Footnotes" section of the "Resources" chapter. That section describes how to incorporate concrete details and improve the layout and attractiveness of your figures, tables, and writing.

Again, keep in mind that your instructor may not make any specific recommendations for improvement, but that does not mean that your word choice and punctuation are perfect or that your argument, analysis, and evidence do not still need improvement. Similarly, do not assume that your instructor has identified everything in your submission that needs to be corrected. The comments, feedback, and grade that your instructor provides are merely representative of the kinds of improvements that you need to make.

I have one more point. Your instructor likely has only a certain amount of time to review your submission and give you feedback. It is almost always true, no matter how excellent your submission is, that your instructor can find meaningful ways to help you improve your work. That means that, each time you submit your work for feedback, it should be your best work so that your instructor can help you improve. If you turn in something that is less than your best, the feedback you get from your instructor will be wasted because it will only be suggestions that you could have identified if you had just taken more time.

2.2 Transition Paragraph

You will need to craft and add language that appropriately helps your readers follow your transition from the Objective section to the Criteria section. One way to do this is to create section headings. If you have not already, go back to your Objective assignment and identify the start of your "Context" section and insert at the start (i.e., just after your methods paragraph) a section heading titled "Context". Section headings should be left-justified on the page and should be distinguished from the rest of the text (e.g., using extra spacing, using bold or underlined text, and/or making the font sizes larger). Similarly, create a section heading for this section of your analysis, titling it the "Criteria" section.

While a few thoughtfully chosen section headings can help readers better follow your analysis, I want you to provide a transition paragraph that adds even more clarity. Here is the basic text that I want you to use:

The first step in MCDA is to identify a set of relevant criteria that the analyst will use to evaluate whatever options or solutions are later proposed to answer the decision making question. In this section, the author names and describes ____________________ [insert the number of evaluation criteria you have in your analysis] relevant decision criteria and the relative importance of each. This set of decision or evaluation criteria seeks to reflect the decision maker's [or, if more appropriate, insert "decision making group's"] underlying interests as completely as possible. When feasible, the analyst defines a plan for how the analyst will objectively measure the level of each criterion. The analyst concludes this section with a proposal and rationale for how to weight the relative importance of each of the evaluation criteria.

Keep in mind that the words in brackets (i.e., "[" and "]") are simply messages to you should not be copied and pasted into your work. Similarly, you will have to insert into the blank spaces (e.g., "__________") the details that are particular to your individual analysis.

2.3 Evaluation Criteria and Their Purpose

This assignment and this section of your report requires you to identify a set at least three but not more than five evaluation criteria. The criteria that you identify should reflect features or attributes that the decision maker thinks are of greatest importance. This set of criteria must involve tradeoffs.

For example, in the Overview chapter, you considered a situation (see here) where a retail shopper was trying to decide which kind of banana would be best to buy given the shopper's preferences and current circumstances. In that example, the shopper identified six evaluation criteria: price, smell, origin, shelf life, cost/weight, and quality. These are six criteria that are important to this shopper, and it is these six criteria that the analyst will use to evaluate and eventually rank a set of options to be determine later (in the next assignment).

In short, you must identify a set of evaluation criteria that best reflect the interests that your decision maker thinks are most important when deciding on an answer to the decision question.

The distinction between "criteria" and "options" in MCDA is similar to the distinction in negotiation theory between "interests" and "positions" [2]. In negotiation theory, a "position" defines a negotiator's stance on specific subject or in a specific situation (e.g., "There is only one orange left, and I have to have it."). An "interest" defines a negotiator's underlying motivations, values, or incentives (e.g., "My recipe requires the juice of one orange," or "My recipe requires the zest/peal of one orange."). Some negotiation experts say that it is useful to focus first on everyone's interests because it open up more opportunities for the parties to identify creative options (e.g., "Rather than cutting the orange in half and both not getting everything we need, let's share the last orange since you need only the juice and I need only the zest."). The focus of this assignment is on what the decision maker is trying to accomplish (i.e., on what criteria or interests are important) and not on how that should be accomplished (i.e., not on what is the best option to pursue or best position to take).

The evaluation criteria you identify must be important to the decision maker, but they must also involve inherent tradeoffs. For example, in the banana example above, the banana that ranks highest in terms of quality is not likely to also rank best (i.e., lowest) in terms of price. A higher quality banana is likely to be relatively more expensive, and a lower quality banana is likely to be relatively less expensive. Similar tradeoffs likely exist between the other evaluation criteria used to identify the best banana.

It may take you some practice and thought before you are able to identify criteria that are both the most important to your decision maker and ones that also involve tradeoffs. Some examples may help you see the distinction between evaluation criteria and options.

• Bananas may be selected based on price, smell, origin, shelf life, cost per pound, and quality.

• Job opportunities may be characterized by salary, location, prestige, and advancement opportunities [3].

• Military weapon systems may be distinguished by their reliability, cost, availability, accuracy, and yield [4].

• Animal feed may be differentiated by its nutritional value, cost, and shelf life.

• Investment portfolios may involve tradeoffs between expected risk, returns, and liquidity.

• Input suppliers may differ in terms of product quality, delivery time, and service quality [5].

• Employee quality may be measured by sales volume, attendance, guest service, communication, and appearance [6].

• Product designs may involve tradeoffs in cost, durability, functionality, size, and delivery time.

A job opportunity that scores highly in terms of salary (e.g., high salary) might likely score low in terms of the safety or quality-of-life of the location (e.g., a hardship post in the foreign service receives higher pay). A military weapon system that is highly reliable (good) may also be relatively costly (bad). An investment portfolio that has a higher expected return may require the decision maker to accept more risk or volatility. An employee that has the highest sales volume may not be more stressed and quick paced and therefore rank lower in terms of friendly and prompt communication. Consumer product designers (e.g., of cell phones) must decide what combination of features, size, functionality (e.g., battery life), and total cost are best and the smallest device may rank highly in terms of size (e.g., very portable) but then rank very low in terms of functionality (e.g., small battery).

Keep in mind that some criteria may be really important to your decision maker but may not involve any tradeoffs. If so, those criteria can be ignored. For example, suppose that your friend who is looking at different job opportunities says, "Whatever job I take, it can only be with a firm that has a non-discrimination policy regarding sexual orientation." Clearly that is a decision criterion that this person highly values, but it is not one that involves tradeoffs because this person has specified that all possible options must satisfy this requirement.

The important point is that, in this kind of analytical decision making (i.e., MCDA) , you do not want to jump too quickly to listing options until you have first defined clearly a set of appropriate criteria that you can use to evaluate any potential option. The set of criteria that you choose should account as comprehensively as possible for all of your decision maker's interests as they relate to the particular decision task. This too will take some careful thought. You should avoid selecting criteria that measure the same or similar interests.

2.4 Objective Criteria

When you determine that a particular criterion is important to your decision maker, you need to (1) give the criterion a simple name (e.g., "price", "liquidity", or "location"), (2) define specifically what you mean by that simple name, (3) explain why the criterion is important to the decision maker, and (4) describe how, if possible, you will measure the level of that criterion objectively. For this assignment, you will do these four things for each criterion you have. You must have a minimum of three and a maximum of five criteria.

You will describe your first criterion in one or two paragraphs immediately following the prescribed transition paragraph. For each criterion, you might have one paragraph devoted to naming and defining what the criterion is and why it is important to the decision maker with a second paragraph devoted to explaining how you will measure the level of the criterion objectively (if possible) or subjectively otherwise (if necessary).

Let's look at an example. Recall that in the banana example one of the evaluation criteria is the price of the banana. You see in the video the list of criteria nicely described in summary form using each criterion's short title (e.g., "Price"). This short example does not provide a detailed description of what "Price" means, but you must do this in your written assignment. When defining what "Price" means, you might ask yourself is it in dollars or some other currency? Is it the price for a a given volume or a given weight? Is it the retail price or is it the wholesale price that might require purchase of a bulk quantity? These questions point to the need for clarity. Just because you identify "Price" as an evaluation criterion does not mean that the single word alone provides the necessary clarification of what it means specifically. So, in your report, you need to be sure to name not only what the criterion is generally (e.g., "Price") but also to define specifically what you mean by that term. In this brief video, the narrator mentions briefly why the shopper values each of the criteria. In your analysis, you will be need to provide a stronger rationale than what the narrator does in this video. And, as you might expect in a short video, the narrator also does not provide any specific details about how he plans to measure each of the criteria, but you will need to describe specifically how you will measure (preferred) or otherwise assign (acceptable) the level of each of the criteria (e.g., calculate price as the average retail price per pound in U.S. dollars at major grocery stores in the Lexington, Kentucky area).

As you work to identify the appropriate set of evaluation criteria for your decision task, you will need to consider and explain to the reader how you will measure the level of each criterion. Ideally, you should aim to measure the level of each criterion objectively, meaning that you have measured the level of the criterion using your own or another's direct observations and/or you rely on externally-verifiable facts.

Just below, I describe several ways to measure the level of a criterion objectively. Ideally, you should aim to measure all of your criteria objectively, if possible. Sometimes that is not possible. So, farther below I present some ways to assign a level to a criterion subjectively, meaning the assigned level is based on personal opinions, interpretations, emotions, and judgements.

Remember, for the Criteria assignment that you turn in to your instructor, you must have at least three criteria that are measured objectively. You may have as many as five criteria total, two of which may be assigned levels using subjective methods.

Direct Measure. Price is an example of a criterion that can be directly measured. Prices are measured in a particular currency for a particular amount of a given product (e.g., $2 for one pound of fresh non-organic bananas). When price is a criterion, you can typically observe the price directly and use a single unit of measurement (e.g., U.S. dollars) for all the possible options (e.g., nearly ripe organic bananas, over-ripe non-organic bananas, and damaged non-organic bananas). Criteria that can be directly measured are convenient because they can be typically measured relatively easily and objectively. Other criteria that can probably be measured directly are:

• Delivery time could be directly observed and measured in minutes

• Sales volume could be directly observed and measured in units per time (e.g., dollars of merchandise per week)

• Stock returns could be directly observed measured in dollars per time (e.g., dollars per year) or percent change in some base amount

• Salary and profit could be directly observed and measured in much the same way as returns

When a criterion can be measured objectively using a direct measure, you probably only need one or two sentences to describe your measurement plan for that criterion. In the banana example, I might write:

The analyst plans to measure the level of the "Price" criterion of a given option by averaging the retail price per pound in U.S. dollars at the three grocery stores in the Lexington, Kentucky that the decision maker most frequently shops.

Or, if I were measuring delivery time as a criterion as part of an analysis of which local delivery service would be best, I might write:

To measure the level of the Delivery Time criterion, the analyst will place a delivery service request with each delivery service for the same day and time and measure how many minutes it takes for each firm to complete the assigned test delivery request. All test deliveries will be local. The analyst will attempt to initiate all delivery requests as close to the same time as possible to ensure that prevailing traffic conditions apply equally to each firm. No adjustments will be made to the time measure even if the firm sends a message indicating a delay, but the analyst will note such occurrences for reference purposes.

Sometimes there might be more than one reasonable way to measure some criteria directly. It might not be obvious which which direct measure is. In these instances, one approach is just to select the most reasonable way to measure the criterion and then provide a justification for your choice, including if needed a clear statement and justification for any reasonable assumptions you might have made. For example:

• Shelf life could be defined to mean "the average number of days from receipt of the product until the stated expiration date"

• Liquidity (e.g., of an asset) could be defined to mean "the average number of days to convert a typical unit into cast"

• Attendance could be defined to mean "the typical number of days per year of work missed"

• Accuracy (e.g., of a military weapon) could be defined to mean "the average deviation in meters from the intended target"

• Size (e.g., of a cell phone design) could be defined as the total square centimeters of the largest single surface

Proxy Measure. As just shown above, you can define what a criterion means, within reason, and how you intend to measure it. Sometimes it may be less clear how to define and/or measure a particular criterion. In some cases, you may define a criterion in a certain specific way, and then acknowledge (to the reader) that there is not a viable way to measure the criterion directly as defined. If this is the case, you might propose to use a a proxy measure.

A proxy is a thing that can be used to represent the value of something else in a calculation. It is an indirect measure. For example, suppose that you want "quality" to be one your evaluation criteria. You could first define "quality" in whatever complex way that you want and then you could explain to the reader why it would be hard to measure potential options using that criterion if defined in that particular precise but complex way. Then, tell that reader that you propose a to use a proxy measure that, for reasons that you must explain, you believe still measures the criterion's level relatively accurately but in a much more convenient way. When you use a proxy measure, you basically admit to the reader that your proxy measure is not a perfect measure but that the proxy measure is relatively adequate given how much easier it is to perform the measurement.

Consider again the example of measuring delivery time as a criterion from above. It might be too expensive or complex to measure directly the delivery time as proposed above. In that case, I could define a proxy measure of the Delivery Time criterion. I might write:

To measure the level of the Delivery Time criterion, the analyst will examine the latest 20 Yelp reviews for each delivery service option and calculate the percent of those reviews that identify delivery time positively. Each negative comment about delivery time will offset one positive comment except that the lowest possible level will be zero percent. If the delivery service option has fewer that 20 total Yelp comments, the analysist will calculate a percent based on the total number of comments available. If no Yelp comments are available for the delivery service option, the analyst will assign that option a level of zero percent for delivery time. The analyst will exclude all Yelp comments that are older than one year.

In the above example, I am using Yelp reviews to measure delivery time indirectly (i.e., using a proxy). Keep in mind that this example does not include the necessary language that justifies why the proxy measure is an appropriate compromise rather than using a direct measure. You will want to include that in your report.

Composite Measure. Sometimes you may not be able to identify a single proxy measure for a particular criterion. In such cases, you may be able to measure the level of the criterion using a composite measure. A composite measure uses a formula to combine data from two or more proxy measures.

For example, suppose that "location" is one of the criteria you want to use to assess which job opportunity is best for a particular decision maker. The benefits of a particular job location obviously will depend on the subjective preferences of the decision maker. Suppose this decision maker tells you, "The best job location for me would be a location that has low housing costs and lots of sunny weather." In this example, you have two factors that determine the level of the "location" criterion.

Having identified these two components (i.e., "composite factors"), you need to propose and define a reasonable way to combine them into a single statistic or number. To be clear, since you have not at this point in your paper identified any specific options, you do not actually need to calculate a composite measure for anything. You will do that in your "Analysis" assignment. In the "Criteria" assignment, you merely need to describe and justify the features of the composite measure.

A composite measure of something (e.g., the value of a particular work location to a job seeker) is the weighted combination of two or more normalized values that measure relevant components of the whole. What does that mean? In this context, normalization means adjusting the measured values to have a notionally common scale. Think about average home prices across different U.S. cities [7]. These home prices cover a broad range from tens of thousands of dollars in low-cost cities to millions of dollars in high-cost cities. If we normalize these values, it means we force the range of prices onto a common scale (e.g., zero to ten). So, that would mean that the city with the highest average home price would have a value of one and the city with the lowest average home price would have a value of zero. The home price values for all of the other cities would be proportionately distributed between the minimum value (0) and the maximum value (10) of the specified common scale (i.e., zero to ten). The same normalization process could be done for some measure of sunshine across U.S. cities [8].

When you normalize a data set, it squeezes all of the data from smallest to largest into a common scale like zero to ten. So, if we normalize the average home price data, we have for each city in the data set a value between zero and ten where values closer to ten represent the more preferred cities (i.e., cities with lower average home prices) and values closer to zero represent the less preferred cities (i.e., cities with higher average home prices). Likewise, if we normalize the number of sunny days per year for each city in a data set, it means that after normalization we have a table showing a value between zero and ten (i.e., a common scale) for each city where cities with value that are closer to ten are sunnier and cities with values that are closer to zero are less sunny.

The last step to calculate a composite measure is to weight each of components, a process that requires you to identify the fractional percent out of 100 percent that you believe each component in the composite measure should have. The fractional percentages must add to 100 percent. So, in the location example, you might decide after talking with the decision maker that each component (i.e., home prices and amount of sunshine) is equally important. If so, you would weight tell the reader something like:

To measure the location value of a city for this analysis, the author will construct a composite measure of city's location value based on two factors, namely the average home price in the city and the city's average annual number of sunny days. Each factor will be weighted equally (i.e., 50 percent). The analyst will use Zillow's estimates for average home prices for 2021 for single family homes (35th to 65th percentile) that covers 25,000 U.S. cities. Normalization of this data will exclude the 250 most and least expensive cities as they are likely unreasonable job locales. To measure a city's sunniness, the analyst will use historic climate data from the U.S. National Oceanic and Atmospheric Administration (NOAA) that estimates the average number of clear days per year for 270 major U.S. cities. When normalizing this data, only cities in the continental U.S. will be used to define the maximum and minimum values for the most and least sunny cities.

Now, before you miss the point, let me remind you that you will actually do these composite measurement calculations as part of your Analysis assignment, not this assignment. In this assignment, if you are going to use a composite measure, you only need to tell the reader (1) what components will make up the composite measure, (2) what data sources you will use, and (3) what weights you will use.

Expert Measure. Sometimes you may not be able to measure a criterion directly, using a proxy, or using a composite measure. These three measurement approaches are best because they are clearly objective. There is a quasi-objective measurement process that relies on the expert opinion(s) of one (or more) expert(s). The adjective "quasi-objective" means that this measurement process is objective in some ways but in other ways is subjective (i.e., generally inferior). But, for the purposes of your Criteria assignment, you can count the expert measurement method as objective if needed. However, in your paper, be sure that you refer to this measurement process as "quasi-objective".

First, consider one key advantage of the three measurement methods that are clearly objective, namely direct measures, proxy measures, and composite measures. The key advantage of these three clearly objective measures is that the measurement process is repeatable. That means that measurement of the criterion's level is invariant (i.e., does not vary) based on who does the measuring. For instance, if you directly measure the cost of an input (e.g., the specific make and model of a combine harvester) then the level you measure for the cost of that input will be the same as if someone measured the criterion's level following the same process.

Second, hold this idea of repeatability in your mind while I describe how the expert measurement process works. Imagine that you want to advise a decision maker about what marketing plan will be best for the firm owner to implement. Suppose further that you decide as the analyst that one of the evaluation criteria should be some measure of each marketing plan's effectiveness (e.g., the likelihood that the plan will generate 10 new clients per month for the next year). Finally, suppose that you cannot find an appropriate and comparable way to measure this likelihood in a clearly objective way. That would not be a great surprise. Marketing plans can be complex after all. So, what do you do? One option is to rely on the subjective opinions of one or more experts in the field [9]. Here are the steps:

• First, define an objective standard for what an expert is for the relevant context. For example, if you are looking at the effectiveness of different marketing plans, you might define the standard of expertise to be any person who has at least a Bachelor of Science degree in marketing from an accredited college or university.

• Next, identify one or more people with the defined standard of expertise. In your case, since you are probably not being paid for your analysis, you might go to the university's marketing department and ask if there is a graduate student with the requisite credential who you could interview as an expert.

• Third, without adding any more information, present each expert individually or, if you have more than one expert, present the group as a whole with a short, clear, written description of the decision context (e.g., one paragraph) and short, clear, written descriptions of each option (e.g., each marketing plan).

• Finally, without adding any more information, present to the expert(s) in writing a request that the expert(s) assign each option (e.g., each marketing plan) a value (e.g., a number from zero to ten) that reflects the expert's (or expert group's) subjective opinion(s) about the level of each option (e.g., each plan's likelihood that it will result in 10 new clients per month for the next year).

Now, recall that one important benefit of a clearly objective measure is that it is repeatable. Notice in the steps above the efforts to make the process as objective as possible. For instance, you see that you begin by defining an objective standard for what an expert is. In the example given, an expert is defined as someone who has a BS degree in marketing. That is an objective standard. If you asked five friends to sort a room full of people into those that meet this standard and those that do not, all five of your friends should sort the same people into exactly the same categories. That is different than, for example, if the standard defining what an expert is was "someone who the analyst believes is an expert" (i.e., a subjective standard).

There is another important way that the above process is quasi-objective. Note that the last two steps rely only on written descriptions of the context, the options to be evaluated, and the levels to be assigned. All of the instructions to the experts is written down. Relying only on written statements and directions and not adding any other verbal explanation or information makes the measurement process more objective, more repeatable. If all of the relevant communication with each expert is written down, that means that the same exact process could be repeated by another analyst. The idea is that, while the opinions of the experts are subjective, the process of eliciting the subjective options is very prescribed and controlled (i.e., objective). That is why this measurement method is considered quasi-objective.

If you decide to measure one or more criteria using one or more experts, you will need to describe in your text the quasi-objective process that you will use to measure the level of each criterion measured in that way. Also, you will need to include an appendix at the end of your paper (e.g., "Appendix A") that shows the exact wording that you used in your written presentation of information and instructions to the expert(s) to elicit the responses. An appendix has the same kind of information that might appear in a footnote, but you use an appendix when the amount of information is so much (e.g., a half-page if it were put in a footnote) that it looks really awkward as a footnote. An appendix should have a title as well (e.g., "Appendix A. Quasi-Objective Process for Eliciting Expert Option on the Effectiveness of Each Marketing Plan Option").

2.5 Subjective Criteria

For the Criteria assignment, you must define at least three evaluation criteria and you must identify a plan to measure each of those criteria objectively. A measure is objective if it relies on your own or another's direct observations and/or you rely on externally-verifiable facts. A measure is subjective if it relies on personal opinions, personal interpretations, your or another's emotions, or your unverifiable judgements. For this assignment, you may define no more than two criteria that have subjectively assigned levels. Remember, for the assignments in this text, the total number of criteria and options you have must add to eight with at least three of each.

Subjective assessments are useful when there is no reasonable way to measure the level of the criterion. For example, suppose you were helping a friend determine what used car to purchase. Suppose you identified criteria that included the vehicle's power (measured in horsepower), reliability (measured as a proxy using J.D. Power reliability ratings), and cost (measured as total monthly payments for a five-year term). Suppose then that your friend said, "Wait, I really like the color red, and I would love to have my car be red." What do you do?

First, in this example, you should clarify that the friend just prefers a red vehicle, not that a red vehicle is a requirement If color is an important consideration but not a requirement, then you might make color an evaluation criterion. If your friend says that a red vehicle is a requirement and no other color will work, then there is no reason to make color a criterion since there would be no tradeoffs. All options would have to be red.

But, suppose your friend said that color is an important criterion, but not a requirement. Then what do you do? There are, of course, ways to measure objectively various shades of the color red, but would not likely capture this preference that you friend has. In essence, your friend's preference is purely subjective. In cases like this when there is no good way to measure an important criteria objectively, you will need assign the criterion's level subjectively. Your friend has a strong subjective preference for the color of the car. In that case, you might show your friend a dozen or so different potential car colors and ask your friend to rank them from most preferred to least preferred, being sure to tell your friend to assume that all else (e.g., the vehicle's cost, reliability, and power) is equal.

Keep in mind that there are stronger and weaker ways to assign subjective levels to a criterion. When you gather or elicit information to make the subjective determination, you should maintain as much objectivity as possible. For example, in the above example, I suggested that you show your friend a bunch of colors and then ask your friend to rank them according to your friend's preferences. How should you do this? Consider some different ways:

1. Open a web browser, Google "cars with cool colors," and then ask your friend which cars look best

2. Find an online website that shows a bunch of different colors for a new Ford Mustang and ask your friend which look best

3. Show your friend a set of differently colored squares from a auto paint shop and ask which look best

The first way is weak because it would be hard for your friend or anyone to really distinguish the color of the car from the car itself. It would be hard, in other words, to keep all else equal. The second way is better than the first option, but still weak because it potentially conflates your friend's preference for a color with your friend's preference for a car with high horsepower. Among the three, the third way is the best because it keeps as many confounding factors the same (e.g., the size of the color squares) and does not present confounding information when trying to elicit preference information.

In general, avoid asking your decision maker(s) open ended questions. For example, do not ask your friend, "What color of car do you like the best?". The problem is that your idea of what "red" means may be very different than what "red" means to your friend.

Finally, avoid making your decision maker do unnecessary work. You are the analyst. Listen carefully to what your decision maker says. In some cases, you might be able to narrow down the range of preferences based on information that the decision maker has already shared.

2.6 Figures That Support Criteria

Your submission for the Criteria assignment must include one figure to clarify each criterion. You can find general information about how to create and format figures in the Figures and Tables section of the "Resources" chapter. Here I want to focus on some ideas to help you identify the most useful figures for this particular section of your paper.

First, the requirement is to include a clarifying figure for each criterion, not a table. Figures include graphs, maps, and probably most relevant images.

Second, it might help if you understand why this requirement exists. It is for two reasons. First, you are most likely using this text and doing these assignments as part of a course. If so, these figures help demonstrate to your instructor that you know how create and format high-quality figures. But, there is another more important reason. The "Presentation" assignment (i.e., the last assignment described in this text) invites you to cut-and-paste all of your visual aids (i.e., your figures and tables) into some presentation software (e.g., PowerPoint) and then add audio to create the required 10-minute long video. That means that your visual aids are (and will be) the visual scaffolding that you use to tell the story of your analysis. That means that you want to think carefully about what figures will be the most useful and relevant as you tell readers in your paper and listeners of your video what you are doing in this section of your analysis. At the end of the Criteria assignment you have a required table that nicely summarizes all of your criteria, their weights, the minimum and maximum values of each, and how you plan to measure them. So, you do not need to repeat that information in these required figures.

Third, the figures that you include in this section of your paper--like all figures--should have the highest information value as possible. That means that each figure should provide useful information that does not replicate information that is already in the text. Your figures should convey information that clearly benefits from a visual presentation. While it is a requirement to include a clarifying figure with each criterion, you should work to find images that are still useful.

Okay, so what useful information might these figures show? My suggestion is that you think about what you would want to say and show in a presentation about your set of criteria. In the prescribed outline, I suggest that you depict somehow the reasonable range (i.e., high and low levels) of each of the criteria.

For instance, suppose you are analyzing what city is best to work in after graduation where sunniness (or probability of a sunny day) is one of the evaluation criteria. You could look up the two cities that are most and least sunny (i.e., the minimum and maximum preference) and juxtapose images of those on a typical day for those two cities (Figure 1).

Finally, I suggest in general that you avoid clip art or other iconic representations when they have no useful information associated with them. For instance, suppose you are analyzing what vehicle is best to purchase after graduation and one of your criteria is the cost of the vehicle. It would not be a good idea simply to have a figure depicting a large dollar sign or a picture of a dollar bill. However, some color is nice to have with any figure. Some symbol or clip art even could useful if they are combined with other brief summary information. In this case, for example, you might create a figure that has that splash of color (i.e., the green dollar signs) but also have some useful supporting text (Figure 2).

In Figure 2 above, notice that what is being conveyed is essentially the same kind of information that is being conveyed in Figure 1. In both cases, the figure shows the range (i.e., high and low) for the particular criterion. Figure 1 shows the range for the sunniness criterion by depicting a city with high sunniness (Yuma, AZ) and a city with low sunniness (Juneau, AK). Figure 2 shows the cost range of vehicles but without using pictures. Figure 2 creates a mental image of the low cost vehicle (e.g., it is a used vehicle that has high mileage, some rust, and is mechanically sound). Can you picture that? Figure 2 similarly creates a mental image of a high cost vehicle, namely a vehicle that is new, is still under warranty, has higher insurance costs, and is relatively reliable. The goal of these two figures is the same. Both figures are trying to provide useful information to readers about each criteria's range of possibilities. The goal is to help readers understand each one of these evaluation criteria better. That should also be your goal as you select one figure to accompany each of your criteria.

2.7 Simple Additive Weighting Method

After you have defined your set of evaluation criteria, you then need to give your reader some additional information about how you will use these criteria and the different levels of each in your MCDA. Specifically, you need to describe the simple additive weighting (SAW) method of MCDA and evaluate how well or not your set of evaluation criteria conforms to an important assumption of that method, namely preference independence.

You will probably start this section of your assignment with a description of the SAW method. MCDA is one kind of decision analysis method or subdiscipline among many in the general field of operations management. Within the subdiscipline of MCDA, there are many different approaches and techniques [10]. The simplest and probably the most popular MCDA approach is the simple additive weighting (SAW) approach. See the Overview chapter footnotes for published examples. While the SAW method is simple, it relies on a fairly constraining assumption, namely preference independence.

Preference independence means that the value, importance, or priority that a decision maker places on any one criterion is not affected by the level of any of the other criteria. In practical applications, when a decision maker’s preference for one or more evaluation criteria is correlated or interdependent, the simple use of criteria weights can distort the interpretation and meaning of the final scoring and ranking of the options. Fortunately, this problem is one of degree. Where the interdependence is relatively minor, the distortions in the final scoring and ranking of options is also typically minor.

There are MCDA methods that avoid the SAW method's constraining assumption of preference independence. However, those other MCDA methods are more complex and time-consuming to implement, they typically utilize computer software, and they often require larger amounts of data. You probably have de minimis financial resources and only one semester of time to devote to your analysis. You probably only have access to public and freely available data. For these reasons, you will need to use the SAW approach for your MCDA, even though your set of evaluation criteria may not conform completely to the preference independence assumption that underlies that method.

Okay, let me stop just a moment. Recall that this section of your assignment submission (1) should be about two paragraphs long and (2) you may use and adapt any of the work from this text, including specific wording, provided you have included the prescribed footnote from the Objective chapter that sets alternative expectations regarding the originality of your work vis-a-vis the work I did for this text. In other words, because this section of your analysis is somewhat more technical, you might want to use or adapt wording from this text (e.g., the three paragraphs immediately above) to use in your assignment submission. You should be able to adapt and combine the three paragraphs above into one single paragraph that accomplishes the first goal of this section, namely describing the SAW method and the preference independent assumption. In the second paragraph of this section, you need to evaluate how well or not your set of evaluation criteria satisfies the preference independence assumption (see below).

Now, let me get back to explaining what preference independence is. Here is an example where the assumption of preference independence might be a problem. Suppose your friend just graduated from college and needs to decide what combination of new job and vehicle are best for her to pursue after graduation. Suppose you have identified two job options (rural land surveyor or city school teacher) and two vehicle options (a SUV or a sedan). In this case, it is almost certain that this decision maker has interdependent (i.e., not independent) preferences. If you recommend that your friend pursue the rural land surveyor job, your friend will prefer a vehicle with off-road capabilities (i.e., the SUV). If you recommend that your friend pursue the city school teacher job, she will prefer the more comfortable ride and fuel efficiency of the sedan. In short, your friend's preference for a vehicle is related her job preference. There is preference interdependence or we could say that the preference independence assumption is violated in part.

Here is an easy test to see how well a set of evaluation criteria satisfies the assumption of preference independence. Pretend you are the decision maker and ask yourself, “To what degree, if any, is my preference about the level of any one criterion (e.g., that its level be high or low) dependent on the level of any other criterion?" Consider an example.

Suppose you are trying to help someone determine what job opportunity is best to choose upon graduation. Suppose you have identified three evaluation criteria. One criterion considers the decision maker's preference to live and work in a city that has lots of sunny weather. The second criterion considers the decision maker's preference to live in a city with a low cost of living. The third criterion considers the decision maker's preference to have a job that pays a high salary. To apply the preference independence test, you want to pretend that you are the decision maker and ask yourself the one question (above) about each criterion.

• Sunny weather criterion. For this criterion, the best possible situation would be sunshine during all daylight hours and no clouds ever. The worst possible situation would be clouds all the time and no sun ever. Now you ask yourself, does your preference for sunny weather change depending on the city’s cost of living (e.g., being higher or being lower)? And, ask yourself, does your preference for sunny weather change depending on your salary level? In both cases, probably not much, but maybe some. If the cost of living is really high, you might feel stressed about paying your bills, thus changing the importance of your salary level. With a high cost of living and a low salary, that might you not care as much (or maybe you would care even more) about how sunny the weather is.

• Low cost of living criterion. For this criterion, the best possible situation would be a really low cost of living and the worst possible situation would be a really high cost of living. Now you ask yourself, does your preference for a low cost of living change based on how sunny the city is or based on how high your salary is? On both accounts, probably not, but maybe so. At most modest levels of salary and at most modest costs of living, the preferences would not be related. It might be that as the city’s cost of living approaches an upper extreme, you would be increasingly concerned that a very low salary would not make living in the city enjoyable, particularly if the city was not very sunny.

• High salary criterion. For this criterion, the best possible situation would be a really high salary and the worst possible situation would be a really low salary. Next, ask yourself, does your preference for a high salary change if the city is particularly sunny or particularly cloudy? Does it change if the city has a particularly high or low cost of living? Again, probably not much, but maybe somewhat. At most, preference seem only partially interdependent. If the job has a particularly high salary, you might be willing to accept a particularly less sunny location. Or, maybe not. Maybe with a higher salary, you would feel more depressed living in a relatively cloudy city, thinking constantly that a person with such a high salary should not have to go without sunshine so often.

After you provide a description of the SAW method, you will need to address in your assignment how well or not your set of chosen criteria satisfies the SAW assumption of preference independence. Remember that the recommendations from your MCDA will be sounder the more completely the preference independence assumption holds. So, I suggest that you look for an example among your criteria where the preference independence assumption might not hold, explain how there might be preference interdependence, and then conclude that the potential interdependence among your set of chosen criteria appears to be relatively insignificant. In other words, you tell your reader that preference interdependence does not seem to be so great a problem for your analysis that the SAW method is not still appropriate. You show the reader where it might be a little bit of a problem and, in doing so, you show the reader--rather than just telling the reader--that the assumption of preference independence is at most only a minor problem for your analysis.

If you believe that there is lots of preference interdependence among your criteria, you should not use the SAW method. But, since this textbook does not describe any other method, you should instead see if you can transform one or more of the offending criteria if you think preference interdependence among your set of criteria is significant.

2.8 Assigning Weights to the Criteria

After you identify and define a set of evaluation criteria, you need to determine the relative importance of each criterion to the decision maker or decision making group. The need to assign weights to each criterion a requirement, as noted above, of the SAW method of MCDA. The typical and preferred way to weight the criteria is to assign a fractional percent out of 100 percent to each criterion based on your best judgement about that criterion's importance compared to the other criteria. The fractional percentages for all of the criteria must add to 100 percent.

In your assignment, you will need to provide a rationale for why you assigned to each criterion the weight that you did. Keep in mind that a rationale is a defense or justification for why you did something and often requires that you both explain what you did, why you did it, and why what you did makes good sense.

Finally, you will need to create a table that shows vertically each criterion's short name and horizontally each criterion's weight, minimum (i.e., worst) value, maximum (i.e., best) value, measurement approach, and if applicable the unit for measuring the level of each. Your table should follow all of the formatting and other expectations detailed in the Figures and Tables section of the Resources chapter of this text. Your table should look similar to Table 1.

In the example above, you see that Table 1 has four evaluation criteria listed in the first column. Notice the nice parallel nature of the terms used to identify the set of criteria; they are all single words in this case. In the second column, you see the criteria weights. The weights all add to 100 since they are percents. The weights are an indication of how important each of the criteria are, and this table nicely lists the criteria in order of importance (i.e., "Compensation" is listed first because it is weighted 70 percent).

The next two columns define the reasonable worst possible and best possible values for each of the criteria. This analysis is looking at what job is best, so it makes sense that the "Compensation" is measured in dollars (see right-most column) with the worst reasonably possible compensation being $25,000 per year (i.e., "$, annual") and the best reasonably possible compensation being $90,000. The "Location" criterion is a composite measure. A composite measure is composed of two or more factors that each is or can be expressed in units, but a composite measure itself has no unit. A composite measure still must have a worst and a best value and is typically either 0 to 1 or 0 to 10.

In the table's notes, you see that the "Prestige" criterion is a measure of the hiring firm's market capitalization (i.e., the number of outstanding shares times that market price of one share). Some hiring firms might not be traded publicly and so would not have a share price and thus no market capitalization either. So, the minimum value for the "Prestige" criterion is zero market capitalization. in 2020, Apple, Inc. had the highest market capitalization at $2.254 trillion, so I used that at the uppermost reasonable possibility. Table 1 indicates that the "Experience" criterion will be measured subjectively on a scale of 0 to 10 where a low score means low employer-provided training and advancement potential whereas a high score means a high potential.

Your assignment submission should have a Table 1 that looks very similar to Table 1 above. Your Table 1 should have at least three but no more than five evaluation criteria. At least three of your criteria should be measured objectively (e.g., directly, by proxy, or using a composite measure or expert opinion).

---------------

[1] While you do not want talk in this assignment about specific options, you may need or want to describe how business or operations are currently conducted (i.e., "business as usual") and, in your options assignment, you may have "business as usual" as one of your options. That is okay.

[2] For more information about and a basic introduction to how interests and positions relate to negotiation theory, see the book (pg 31) by Fisher and Ury titled Getting to Yes: Negotiating Agreement Without Giving In (1981).

[3] This and the following example comes from a RAND publication from 1968, but it is very readable and only 78 pages. The sections of the report that relate most to the analysis in this textbook are sections I, II, and the short description of "Additive Weighting" in section III. MacCrimmon, K. R. 1968. "Decisionmaking Among Multiple-Attribute Alternatives: A Survey and Consolidated Approach." RAND Corporation: Santa Monica, CA.

[4] Ibid.

[5] For example, here is an article that looks for the optimal input supplier based on product quality, delivery time, and service quality. George, J., P. Badoniya, an H. A. Naqvi. 2018. "Integration of Simple Additive Weighting (SAW) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) for Supplier Selection." International Journal for Science and Advance Research in Technology. 4: 18-22.

[6] For example, here is an article that seeks to identify an optimal way to select outstanding employees based on sales volume, attendance, guest service, communication, and appearance. Jollyta, D., R. Yunarto, and J. Ridwan. 2018. “Comparison of Analytical Hierarchy Process and Simple Additive Weighting on the Selection of Outstanding Employees." American Journal of Engineering Research. 7: 36-45.

[7] For example, at the Zillow website, I see there is monthly time series data (1996 to 2021) for their estimated value for single family homes (35th to 65th percentile) in more than 25,000 U.S. cities. With so many cities, I decided it was reasonable to exclude the most expensive 250 cities and the least expensive 250 cities as likely outliers. Of the remaining cities, I found that in January 2021 Pinecrest, Florida has the highest valued homes at $1,214,996 (upper bound value) and Lansford, Pennsylvania has the lowest valued homes at $35,871 (lower bound value).

[8] For example, at the U.S. National Oceanic and Atmospheric Administration (NOAA) website, I see there is climate data showing the average number of clear days per year for 270 U.S. cities. The city with the most clear days (i.e., upper bound value) is Yuma, Arizona with 242 days. The city with the fewest clear days (lower bound value) is Cold Bay, Alaska with only 10 days.

[9] When expert measurement relies on a group of experts and focuses on measuring values in the future (i.e., forecasting), the method is often known formally as the Delphi method. The name derives from the ancient Greek high priestess who by accounts forecasted the future at the Temple of Apollo in the city of Delphi. For an early description of the Delphi method, see Dalkey, N., & Helmer, O. 1963. "An Experimental Application of the Delphi Method to the Use of Experts." Management Science. 9(3): 458-467. The summary of this article says that the Delphi method "was devised in order to obtain the most reliable opinion consensus of a group of experts by subjecting them to a series of questionnaires in depth interspersed with controlled opinion feedback."

[10] For example, see Tzeng, G. H., and J. J. Huang. 2011. Multiple Attribute Decision Making: Methods and Applications. CRC press.