Data from Peer Evaluation Surveys

How to use adjustment factor as a grading tool?

First of all, adjustment factors will let you know the relative performance of all members of the team when comparing the scores of all team members (you could see the score to see if anyone over-performs or under-performs compared to other teammates). While the absolute rating score will give you the absolute information of the performance level for each individual based self- and peer- impression. Secondly, if in your class, you care more about the overall teamwork competency, then you might want to calculate the adjustment factors across all dimensions. However, if you care more about certain dimensions of the CATME teamwork competency, you could just calculate the adjustment factor scores for the corresponding dimensions. Again, your calculation and interpretation of the scores should really be based on your needs.

Translating CATME peer evaluation rating into standardized scores. If you have a chance to look over some academic papers related to CATME, you will notice that different people will have different rating patterns and we highly recommend using CATME peer evaluations as the ways to facilitate students' development of teamwork competency. We typically would use CATME peer evaluation results as formative assessment - which means the major purpose of the assessment is to provide an instrument for students to reflect on their past behavior and get feedback from their peers so that they could use the information to further improve.

How do I get rid of ‘release survey’ reminder emails?

Reminder emails can be eliminated by simply releasing the survey results--to students, to researchers, or to no one.

What does “enable extra messages” do?

The CATME Peer Evaluation instrument uses the data it collects from students to recognize certain “exceptional conditions”– dysfunctional teams, over- or under-performing students, etc. When the system recognizes one of these conditions, the system normally provides the affected students with an additional informational message at the top of their survey results. You may choose to have the system NOT display these messages by unchecking this box. 

Do students see Peer-to-Peer Comments when the data is released?

Students will see Peer-to-Peer Comments as long as the “Enable Peer-to-Peer Comments” box is checked. If you do not wish to share the Peer-to-Peer Comments with students, uncheck this box before releasing the survey results.

What results do students see when the data is released?

The students will receive an email inviting them to log into the system to see their feedback. This includes a display of each of the five CATME Peer Evaluation behaviorally anchored rating scales (or just the ones you assigned if you didn’t assign all five dimensions), which includes:

  1. The student’s self-rating – indicated by an arrow in the first column labeled “How You Rated Yourself”
  2. The average rating assigned to the student by the team – indicated by an arrow in the second column labeled “How Your Teammates Rated You”
  3. The average rating on the team – indicated by an arrow in the third column labeled “Average Rating For You and Your Team”
  4. The complete behaviorally anchored rating scale on which the Peer Evaluation ratings were made.
  5. (optional) If you checked the box on the Class Editor page to include extra messages to students, they will see research—based suggestions to improve their performance on each dimension that you surveyed.

You can see the specific feedback results that any particular student will receive by going to the Summary Report Page for the Peer Evaluation. Look for the text that says “Preview results page for student.” (Only available after the survey is released) Click on the drop-down menu to find the particular student whose feedback you wish to preview, and click the “Preview” button.

How do I release the results from a Peer Evaluation survey?

On the Activity Results page, there is a button that says “Release” in the bottom left corner (you may have to scroll down to see it). Above the button is a drop-down menu indicating the different options to release the results to students, to researchers, or to no one. Make sure to select the desired option, then click the “Release” button.

What do the different notes mean?

You may see the notes: under, low, high, over, manip, cliq. These are “exceptional conditions” that the system can identify in teams. For more information, see our Exceptional Conditions Video.

What does “same rating across team members” mean?

Same Rating across Team Members means that a student gave the same rating to every teammate. This could be a sign that the student did not think about their answers, and instead just clicked the same box for every teammate. The numbers will be highlighted in red.

What does “same rating across dimensions” mean?

Same Rating across Dimensions means that a student gave the same rating to a teammate across all of the CATME dimensions. This could be a sign that the student did not think about their answers, and instead just clicked the same box for every dimension. The numbers will be highlighted in red.

What else can I include in dashboard detail by team?

You can include Peer-to-Peer Comments, Comments to the Instructor, and Rater Practice Result.