The decision to accept or deny items can be made after they have been scored. Initially all items appear in the “Undecided” column. Items in each column are sorted from the highest (best) score to the lowest. Any items that remain in the Undecided column by the time the review is closed will be automatically treated as Denied.
If a Target Number to Accept was defined for the Review, then that number is shown on this page and is used to warn if the number of items in the Accepted column exceeds the Target Number. Note: This target does not prevent you from choosing to accept more or less than the original target. For example, if you intended to select twenty items, but items 20 and 21 both received the same score, you may choose to accept both or deny both leaving you over or under your original target after your final decision.
Decisions are automatically saved as they are moved between the three columns.
Individual Decisions
Every item is presented with buttons to move the item to either of the other two columns. If items have been moved into the Accepted or Denied columns as part of a bulk decision, they can be subsequently moved individually to a different column until a satisfying decision is recorded for every item.
Bulk Decisions
Using one of three methodologies, a Decision Line can be placed where desired in the sorted list of items in the Undecided column. The Decision Line lets you easily move all items above it to the Accepted column or all items below it to the Denied column.
The bulk decision option only applies to items in the Undecided column. If you want to reverse a decision to accept or deny an item, the item must be moved individually to the desired column.
Best Practice: Initially only move items to the Denied column that you absolutely know you don’t want to accept. This will distinguish them early from other items that can be bulk moved from Undecided to Denied after all Accepted items are decided and will prevent inadvertently moving undesired items into the Accepted column.
Tip: If you want to move all items from the Undecided column to the Denied column, choose “Top” as the Methodology and set the Value to one more than the number of items in the Accepted column. The Decision Line will not be shown if the break point is at the very top of the Undecided column. By setting the Value to one greater than the break point, the line will be shown and will let you bulk move everything under it to the Denied column. You can easily move the one remaining item at the top individually from the Undecided column to the Denied column.
Methodologies for bulk decisions:
- Top – If you want to select the Top 10 items, specify “10” as the Methodology Value. The Decision Line will be placed below the tenth item in the Undecided column, less any that are already moved to the Accepted column. In this example if there are no items in the Accepted column, the line will appear below the tenth Undecided item. If three items have already been moved to the Accepted column, the line will be placed below the seventh Undecided item.
- Percentage – Percentage (0-100) is calculated for each item based on that item’s score in proportion to the total possible score allowed by the scoring rubric. If the total possible review score for an item is 20, then an item with an average score of 15 will be shown as 75% of the maximum possible score. Setting the Methodology Value to 50% will place the Decision Line below the item having the lowest percentage that is at least 50% of the maximum possible score.
- Percentile – Percentile shows the relative position in relation to the actual set of scores rather than the maximum possible score. The highest scored item regardless of the column it is in is at the 100th percentile. The lowest scored item is at the zero percentile. The 200th highest score in a set of 400 items is at the 50th percentile. This lets you easily select all items in the top half or top 25% of the entire set.
Note: The line will include precisely the designated number of items, so the line may be placed between two items having identical scores. You should look at the scores immediately above and below the line before making a bulk decision.
If a Methodology Value is entered but no Decision Line appears, then no items remain in the Undecided column that are above the Methodology Value. Similarly, if any items in the Denied column satisfy the Methodology Value, they are ignored since a decision has already been recorded for them regardless of their rank by score.
The chosen Methodology and associated Value can be easily changed to try “What If” conditions. If you want to keep the current Methodology and Value for when you return to the Item Decision page, click the Save button. The Methodology and Value can also be set on the Review configuration page.
Show Names
By default personally identifying names associated which each item are hidden from view. This helps prevent bias in a blind decision-making process, especially if the page is shared with reviewers who are conducting bias-free scoring. If there is a need to reveal the identity, the toggle can be used to show the names or to again hide them from view.
Export to Excel
You can download a Microsoft Excel spreadsheet containing all of the items along with their total scores, decision and other relevant information. This enables other options for how you conduct your decision process. You may want to incorporate other information into the spreadsheet such as to combine scores from both application packets and interviews with the freedom to sort in different way to ultimately reach a decision for each item in the list. There is no method to import the results back into the Decision tool. They must be manually selected in the online tool.