Peer Evaluation
View this page as a brochure! Download our Peer Evaluation brochure here.
For videos on the CATME Tools, check out our Instructor Videos page and our Student Videos page.
Peer Evaluation: The Original Comprehensive Assessment
When using teams in education, instructors often use peer evaluations and self-evaluations to assess how effectively each team member contributes to the team. The Comprehensive Assessment of Team Member Effectiveness (CATME) was developed for this purpose. This web-based instrument collects data on team-member effectiveness in five areas that research has shown to be important.
Contributing to the Team’s Work
Interacting with Teammates
Keeping the Team on Track
Expecting Quality
Having Relevant KSAs (Knowledge, Skills, and Abilities)
The CATME Peer Evaluation instrument is a behaviorally anchored rating scale, which describes behaviors typical of various levels of performance in each of the five categories, known as the CATME Five Teamwork Dimensions. Raters select the category of behaviors that most closely matches the behavior of each student on their team (including themselves). Students can practice using the system with fictitious team members in a feature called Rater Practice.
The first of five rating categories: Contributing to the Team’s Work
Understand What is Happening in Student Teams
The system alerts faculty to exceptional conditions, which are rating patterns that warrant attention.
Student results: self, team, average
- Low—a student who rates him/herself as ineffective and who also receives “ineffective” ratings by teammates.
- Overconfident—a student rated as “ineffective” by teammates but rates him/herself as much more effective.
- High—a student who is rated as highly effective according to both teammates and self-ratings.
- Underconfident—a student rated as highly effective by teammates but who under-rates her/himself.
- Manipulator—a student who rates him/herself as highly effective and who rates teammates as ineffective in disagreement with teammates. Such a student may be trying to influence the distribution of grades unfairly.
- Conflict—a team in which there is considerable disagreement among the various raters about the effectiveness of an individual student.
- Clique—a team in which cliques appear to have formed. The ratings show that subsets of the team rate members of their subset high and members of other subsets low.
Some of these conditions have more than one explanation. A student flagged as a “manipulator” might actually have performed a disproportionately large amount of the work on the project even though they worked to engage their teammates in the process. Thus, an instructor’s involvement and judgment are critical when exceptional conditions are flagged. Though the formal study of these exceptions has not been completed, instructors using the system have reported that both the clique and conflict conditions have accurately provided early warnings of those conditions.
Extra Questions
Instructors can also utilize other sets of questions besides the CATME Five Teamwork Dimensions to further understand team dynamics. The extra questions sets can tell you if a student feels safe in their group, if a group has a lot of gossip, and more. See the Extra Questions page for more details.
The CATME Online Interface
The CATME website is a secure interface for collecting data on team-member effectiveness and reporting different views of the data to faculty and students. The CATME system has a number of convenient features—the ability to upload student and team data from files generated by Microsoft Excel; support for multi-section courses and teaching assistants; the ability to edit teams, reset surveys, send email reminders, and track survey completion. The system also allows students to make comments for instructors to read, and provides the option for students to comment about their teammates. It can also compute grade adjustments based on how the ratings patterns compare with faculty-specified criteria.
Instructors can request an account by going to the Account Request page.
The wizard-based interface for class creation in CATME is typical of other setup screens.
CATME Peer Evaluation is part of the CATME SMARTER Teamwork system, which includes other team-support tools. The CATME online interface was developed by Deer Run Associates. This material is based upon work supported by NSF Awards 0243254 and 0817403.
Development and Validity Testing for CATME Peer Evaluation Assessment
Ohland, M. W., Loughry, M. L., Woehr, D. J., Bullard, L. G., Felder, R. M., Finelli, C. J., Layton, R. A., Pomeranz, H. R., & Schmucker, D. G. (2012). The comprehensive assessment of team member effectiveness: Development of a behaviorally anchored rating scale for self and peer evaluation. Academy of Management Learning & Education, 11, 609-630. This paper won the 2013 Maryellen Weimer Scholarly Work on Teaching and Learning Award.
If the CATME terms make you confused, please check the definition on the CATME Terms Instructor Dictionary page.