Task Force Recommendations & Progress
In 2008, the Executive Vice Chancellor and Provost and the Chair of the Berkeley Division of the Academic Senate charged a campus-wide Task Force on Teaching Evaluation to make recommendations on evaluation practices across the Berkeley campus. At the end of its year-long study, the task force published a report that summarized its findings and included eight key recommendations for follow-up work. Below is a summary of each recommendation and the campus’ significant progress in implementing them since the submission of that report.
The UC Berkeley campus has taken a systematic, thoughtful and deliberate approach to understanding the campus needs for course evaluations under the direction of the Online Evaluation of Courses (OEC) Steering Committee. Since the inception of the initiative, the project team has regularly consulted the Committee on Teaching (COT) and created diverse campus-wide working groups to advise, to define, and to give feedback and direction. Activities engaged in with these groups include focus groups, review teams, individual information gathering and feedback sessions with instructors, students and department staff, and regular Steering Committee Meetings.
Recommendation 1:
End-of-term evaluations completed by students should meet the following criteria:
- Uniformity: a common subset of data for all instructors
- Transparency: forms should provide valid and reliable information on instructor effectiveness
- Flexibility: departments should be able to tailor their forms according to their own needs and practices - adding questions of interest as appropriate
Work completed: In consultation with over 50 faculty and under the direction of the Online Evaluation of Courses (OEC) Steering Committee, the project team developed 13 pilot instructional formats, designed to provide a more uniform set of questions concerning course and teaching effectiveness, while respecting the specific needs of different teaching/course formats and disciplines. These formats have been piloted for the past two years, and have informed our design for templates that will be used in the new online system (see below).
Recommendation 2:
The following information should be gathered in the end-of-term evaluation process for every course in every department:
- Numerical evaluation for instructor’s overall teaching effectiveness, as per memorandum from Vice Provost de Vries on 11/19/02
- Numerical evaluation of how much the student learned in the course
- Self-reported level, major, and GPA of the student completing the form
- Departmental averages and standard deviations for overall teaching effectiveness
- Grade distributions for the course and the average grade distributions for comparable courses
- Student written comments
Work completed: The end-of-term course questions used during the pilots retained the required numerical questions pertaining to instructor and course effectiveness. They also include open-ended questions that encourage written comments. Both will continue to be a feature of the questions used in the new online system. The project team is also actively investigating integration with the campus’ s enterprise data warehouse (EDW) to enable data from end-of-term course evaluations to be linked with GPA, major, and other key data that will provide a fuller contextual picture of a students’ ratings and of students’ experience across multiple courses.
Recommendation 3
Information other than student ratings should be provided for evaluating teaching in merit and promotion cases:
- A self-statement that is included in every case for merit and promotion, and that specifically addresses teaching
- A statement from the department chair that comments on the teaching effectiveness of the faculty member (The chair’s statement should contain information and observations beyond what is found in the end-of-semester evaluation forms)
- Syllabi for the courses that show the content, grading philosophy, and assignments
- For mid-career appraisals, peer observation of a faculty member’s teaching
Work completed: In the spring 2013 semester, the Center for Teaching and Learning (CTL), the Berkeley Division of the Academic Senate Committee on Teaching (COT), and the Department of Statistics collaborated on a pilot of peer review of course instruction. Resources and forms to guide the peer review process were created by the CTL, with input and approval from the COT. The success of the pilot has led several other departments to pilot the same peer review process and has spurred conversations in the Academic Senate about ways to incorporate more consistently peer review into the merit and promotion process. This is a significant step in broadening the types of evidence used in merit, tenure and promotion cases beyond end-of-term course evaluations. The CTL also offers resources on developing teaching portfolios that incorporate multiple types of evidence of teaching effectiveness.
Recommendation 4
The campus (through the Office of Educational Development) should develop a database of good end-of-term evaluation questions that can be made available to all departments; departments should be encouraged to revise and streamline their forms, as appropriate, and the campus should provide departments with consultation and support in revising their current forms.
Work completed: The campus has recently (as of 10/2013) implemented a new online system. The system allows for the creation of a question bank. As the system’s use across campus expands, the question bank will grow.
Recommendation 5
The Administration, Academic Senate, and departments should strongly and consistently urge faculty to use mid-semester evaluations and other methods of early feedback in their classes.
Work completed: The CTL website provides extensive resources, recommendations and examples of ways faculty can utilize mid-semester evaluations and other forms of formative feedback early in classes to gauge student learning and make necessary adjustments in pedagogy. These resources are actively disseminated and promoted in CTL face-to-face programming, including the Teaching Excellence Colloquium for faculty new to Berkeley, and through electronic communications, including timely reminders through the Teaching@Berkeley newsletter.
Recommendation 6
The campus should fully support efforts to pilot a separate system that will provide more relevant course evaluation information to students.
Work completed: As part of the multi-term online evaluation pilot, the OEC steering committee recommended that student-to-student questions be integrated with end-of-term surveys, and these questions have been tested in the pilot. Departments will have the ability to decide to include and share the student-to-student questions. The adopted system is flexible enough to accommodate this option.
Recommendation 7
The campus should pursue and fund the development and implementation of an online or hybrid evaluation system. To accomplish this, the Task Force recommends appointing a small working group with technical and pedagogical expertise to recommend the selection of a centralized system. Such an evaluation would include addressing policy, legal, and procedural issues.
Work completed: A project team, under the direction of the Online Evaluation of Courses Steering Committee, has run four semester-long pilots, examining response rates, response bias and quality of open-ended responses. The reports from the most recent pilots include the 2012-2013 Progress OEC Progress Report and the 2011-2012 Midterm Progress Report. In Summer 2013, the campus completed an RFP for an online course evaluation system. The system will be used this Fall with 3 large departments, with additional adoptions coming in Spring 2014. Use of the system will be on an opt-in basis at the department level. Once we have moved several departments over to the new system, we will open the system up for use in optional mid-term evaluations as well.
Recommendation 8
The campus should provide departments with information and consultation services to facilitate the recommended changes in evaluation.
Work completed: The CTL and ETS, with the guidance of key Academic Senate committees, are developing this program and providing departments with necessary support consultation and information.
Please check the ETS Course Evaluations project site and Service Site for additional information and updates. For questions and comments please contact course-evaluations@berkeley.edu.