Rubrics are a special type of form used to accept scores for Reviews. Elements are limited to radio-type response options for selection of an integer score in a specified range such as 0-5 or 1-10. This special type of form must be initially configured within the Reviews administrative module.
The rubric form will be displayed over the top of each item presented for review (the item can be scrolled under it) so it will always be visible regardless of how much content is being displayed for the item under review.
Due to the amount of information displayed in a Review (including both the item being reviewed and the overlaid scoring rubric), it is recommended that screens at least 13″ in size (measured diagonally) be used for Reviews.
Two main factors can be configured for a rubric form:
Criteria
Criteria used as the basis for scoring can be fully customized. It is recommended that the number of criteria be limited since each criteria along with its associated scoring scale take up screen space that would otherwise be used to present the content being evaluated. Eight criteria would take up about half the vertical space on the screen (assuming the scoring scale for each criteria can fit on the same line). It is recommended that scoring generally be limited to no more than five criteria.
You may specify whatever criteria is appropriate to make a wise decision. There may just be one “Overall Recommendation” criteria. There may be one for each question being scored on an application. There may be one for each value such as “Clear Objectives”, “Topic Relevance”, and “Audience Engagement” to score proposals for workshop presentations. Or the criteria may be a hybrid combination of these possibilities.
Scoring Scale
Responses may be defined as a sequence of integers such as 0-5 or 1-10 with the highest number being the best possible score. The scale should be sufficient to make an acceptance decision for the expected number of items undergoing review. A 0-1 scale may be sufficient for a simple No/Yes decision whereas a 0-10 scale may be necessary to evaluate the quality of someone’s relevant experience.
Each criteria can use a different scale. The upper limit of the scale determines the relative weight of the criteria. A scale that goes up to 10 has twice the weight of a scale that only goes up to 5. The scores of all criteria will be added to provide the total score for an item.
Since it is possible different items being scored may have a different number of reviewers, the averages of the total scores for each item are used to rank the items from highest (best) to lowest.
Best Practice: It is recommended that the upper limit of the scale be no more than 10. The rubric form can typically display 12 options before the options wrap around to a second line. Thus 0-10 (11 options) would fit in a single row on a typical laptop or desktop monitor.
Best Practice: Separately provide all Reviewers a guide to normalize their scoring. For example, if using a 0-8 scale, does a zero represent a very bad answer or no answer at all? Is a “4” expected as an average score? If so, what does an “average” quality item look like? If using a 0-3 scale, the zero may mean “Fails to meet expectations” while 1-3 mean “Meets”, “Exceeds” or “Significantly Exceeds”. Or 0-3 scale could mean “Definitely No”, “Probably No”, Probably Yes”, and “Definitely Yes”.