4(b). Program Review
2.7 (program review), 4.4 (program approval), 4.6 (evaluating for improvement), 4.7 (improvement of evaluation methodology), Questions 1 (ensuring alignment with purposes) and 5 (reviewing and modifying programs) under Teaching and Learning (2)
The program review process is critical to improve and maintain the high quality of our academic programs. The earliest campus academic reviews at the University of California date from the 1960s, and the first official review on the Berkeley campus as we know the process today was conducted in 1971. Formal recognition of Graduate Council responsibility for reviewing programs for their quality and appropriateness was made in October 1976, with the addition of By-Law 106 of the System-wide Academic Senate. Since the 1990 WASC review, we have completed reviews of 47 out of approximately 90 departments. Thirteen more reviews are active and not yet concluded. We have also completed reviews of 23 out of 26 graduate groups, with two more currently in progress, and we have piloted a new review process for our non-departmental undergraduate programs.
Before taking an in-depth look at our current efforts to improve the program review process, we offer the following case study as a complex example of the way the program review process has functioned to strengthen the quality of our academic programs and to demonstrate our use of educational objectives as guides for decision-making.
1.2 (achieving objectives) and questions 2-4 (reviewing purposes, using evidence, objectives guide action) under Institutional Purposes (1), 2.7 (review and modify program), 4.1 (institutional reflection), 4.2 (aligning needs and objectives), 4.3 (evidence guides planning), Question 1 (evaluation and priorities) under Strategic Thinking and Planning
Background. Berkeley's academic planning efforts have been based on a principle of evolutionary development. Rather than setting forth a categorical blueprint of Berkeley's academic future, planning at Berkeley articulates the conditions appropriate for all faculties and programs to follow emerging trends in contemporary research and scholarship, to update their curricula, to adapt their programs to the requirements of the changing generation of students, and to renew their capabilities through a regular inflow of new talent. The establishment of Berkeley's School of Information Management and Systems (SIMS) illustrates how this principle of evolutionary development works.
SIMS's genesis can be traced to a regular Graduate Council Review of the former School of Library and Information Studies (SLIS) in 1989-90. SLIS's domain was the transmission and use of recorded knowledge. It was accredited by the American Library Association, and most of the members of its advisory board were professional librarians.
In its 1990 report, the ad hoc Review Committee concluded that SLIS was an important and national resource in the rapidly growing field of library and information studies. However, the committee observed some troubling signs and symptoms in the School as well as other library schools. It found, for example, that SLIS had suffered from a lack of academic and research leadership for several years, was missing a forward-looking statement of educational objectives that would lead to a reasoned academic and research plan, had few academic or research linkages with other parts of the campus community, and had a small extramural research program. The Committee recommended that the faculty and a new dean, when hired, reexamine their goals, make a serious assessment of future directions of the professional field, and develop a vision of how the School could become a leader in the field.
Exploring a New Vision. The faculty were asked to prepare a vision statement, describing how they imagined the development and evolution of the School's academic and research programs during the next decade. The faculty submitted a Vision Statement to the campus in December 1991. The overall goals were to expand the School's focus on information systems of all types, including libraries, increase its research base, and improve its research and educational links with the rest of the campus. It also recommended that the School continue as an independent unit but that its name be changed to School of Information Studies.
A Special Evaluation Committee reviewed and evaluated the vision statement and the structure of the School, its relation to the rest of the campus, and its continuation in its present form. The committee's recommendations, completed in April 1992, were framed when many library schools and departments were closing throughout the US. It was a period that saw substantial and rapid changes in the technology and in the nature of the profession. With this as backdrop, the committee affirmed that the field of information studies held great promise and should have a presence at Berkeley. To achieve that goal, it would be necessary to recruit a new dean with a clear, well-focused vision. Should such leadership prove unavailable, "only then should [the School] be reexamined with serious consideration being given to its permanent closure (Evaluation Report)."
The Graduate Council Review agreed with the Evaluation Committee's recommendation that the status quo was unacceptable: "The preferred result is to pursue rebuilding the School along the lines recommended by the Evaluation Committee." This, of course, required considerable investment of resources. The question of whether it was possible to move forward with rebuilding, within the unstable University and state budget climate of the 1990s, was referred to the new Academic Planning Board for consideration.
Academic Planning Board Interventions. The Academic Planning Board was established in Spring 1992 to implement an impartial long-range academic planning process, given budget reductions. It was also charged with overseeing the process for assessing broad areas of academic work. Its membership was divided equally between Academic Senate leadership and the administration. The Board recommended that the campus recognize the field of Information Science and Information Studies as vital for its mission, and that the present program, organization, and structure of the field on the campus undergo substantial redirection and reorganization. It also recommended that a Planning Group be established to develop a viable definition of the field, identify possible leadership, and locate external resources and support. Finally, it recommended that admission to the School be suspended and campus support of the School be limited to its present level (Resolution, February 1993). Many faculty, students, and alumni of the School considered suspension of admissions tantamount to closure and vehemently objected.
A Wholly New Professional School. Appointed in March 1993, the Information Planning Group was charged to develop, within the constraints of the Academic Planning Board's resolution, a viable definition of the fields of Information Science and Information Studies appropriate to the role of the Berkeley campus. After consulting broadly within and outside the University, the Planning Group recommended a separate professional school with a doctorate and master's degrees "that will advance through teaching and research, the organization, management, and use of information and information technology, and enhance our understanding of the impact of information on individuals, institutions, and society (SIMS Proposal, December 1993)." The Planning Group believed Berkeley had an opportunity to pioneer the development of an emerging professional field of critical importance, an important educational objective and service to higher education.
The challenge for Berkeley was to define at once a new field appropriate to an internationally prominent research University and of critical importance to the state of California. It sought to accomplish this objective by developing a new program uniquely focused on the use and management of information through the merger of technical and social sciences approaches and by addressing applications that cut across disciplinary and organizational contexts. The new program would be concerned with the design and use of information systems and services, and with understanding how people seek, obtain, evaluate, use, and categorize information. Berkeley was an ideal place for this new focus, given (a) strength in such allied disciplines as computer science, business administration, cognitive science, and public policy; (b) the existence of a substantial foundation from the predecessor School of Library and Information Studies; (c) the proximity of leading firms in the information industry; and (d) the campus's ability to attract an eclectic group of outstanding scholars.
The new master's degree awarded by this program would be substantially different from the MLIS degree, reflecting the broader mission of the new School. It was not designed to meet American Library Association requirements; rather, it was intended to serve as a model for the development of accreditation criteria for the emerging discipline upon which the new School would focus. The new School would also sponsor a strong Ph.D. program focused on defining and leading the intellectual development of this emerging field, providing support for the faculty in their creative endeavors, and meeting the strong market demand for such a degree, both in academic settings and in the private and government sectors.
SIMS. In May 1995, the Regents approved the creation of the School of Information Management and Systems and simultaneously disestablished the School of Library and Information Studies (VC Christ memo, June 16, 1995). A new dean (Hal Varian) was appointed in July 1995 and the School was launched in Fall 1995. A number of faculty from SLIS were transferred to the new SIMS. The first class, including both master's and Ph.D. students, was admitted Fall 1997. SIMS has now graduated 110 master's students and expects to graduate its first Ph.D. students next year. Two of the more recent faculty appointments are in the areas of computer networks, internet economics, human-computer interaction, information visualization, empirical computational linguistics, and information access systems. New faculty prospects, in the form of joint appointments with other departments, reflect the inherently interdisciplinary nature of the field, encompassing such areas as law, business, computer science, cognitive science, library/information studies, public policy, and communications. It is also anticipated that the School will offer joint courses with several other units on campus. The context for change was the campus's routine review of its academic programs. In this example, an old program made way for new challenges and opportunities, leading to program renewal and revitalization.
Improving the Program Review Process
As the case study illustrates, the campus's program review process is a way to ensure that the quality of our educational programs keeps pace with the evolution of knowledge. Since the last WASC accreditation, the Berkeley campus has been taking steps to improve and refine its on-going program review process. In 1995, the departmental self-review was instituted as part of the review process. In 1997, the former Executive Vice Chancellor and Provost (EVCP) appointed an Academic Planning Board Working Group to codify procedures, propose revisions to the existing review procedures, streamline the process, and create more coordination between graduate and undergraduate program reviews. The process includes six stages (Program Review Process): (1) the selection of programs/units for review by a joint Academic Senate/administration review selection committee; (2) a unit self-review; (3) a review of the self-review by separate internal and external review committees; (4) a discussion meeting; (5) position papers; and (6) a wrap-up meeting. In 1997 the former EVCP and the former Dean of Undergraduate and Interdisciplinary Studies also recognized that non-departmental undergraduate degree programs and programs in departments without a Ph.D. program were not subject to the usual Graduate Council (GC) reviews. In consultation with the Senate Committee on Educational Policy (CEP), an undergraduate program review process was developed and piloted with the Religious Studies program.
Despite these on-going efforts to address the effectiveness of our procedures for the review of instructional programs and units, both the Academic Senate and the administration continued to be concerned about the lack of timeliness in departmental reviews. Moreover, there was a shared perception that the time and effort invested in such reviews was not always yielding relevant and useful outcomes. Affirming the importance of program review in assuring academic quality, the EVCP convened a special Academic Senate/administration Academic Program Review Task Force in March of 2001 for the purpose of developing specific recommendations to improve the campus's program review process.
The Task Force also investigated the program review process at four comparable institutions (Harvard, Stanford, MIT, and UCLA). In its report (Academic Program Review Report, 2001: main report and attachments B and D; attachment C; attachments A, E, and F), the Task Force made five recommendations to improve the review process: (1) the development of clearer guidelines for the review of undergraduate programs; (2) the establishment of a well publicized timeline for program reviews; (3) better analysis and coordination of the delivery of data to departments prior to their self evaluations; (4) the replacement of the dual external and internal review committees with a single external committee; and (5) the streamlining of the Academic Senate role in the review process.
The Task Force found that the most significant problem with program review was the protracted timetable for completion due to a burdensome administrative structure and a lack of adequate support for departments. Recommendations two through five were aimed at streamlining the process to ensure that reviews were completed within two or at most three semesters. The first recommendation addressed the most complex issue that surfaced in the course of the Task Force's work: the lack of parity between graduate and undergraduate program review. Academic Senate Bylaws spell out the role the GC plays in conducting reviews of graduate programs; however, no similar charge existed for undergraduate program review. Since Academic Senate ownership of the process had resided with the GC, there was an understandable corresponding emphasis on graduate education. The inadequate attention to undergraduate programs was compounded by an absence of specific measures of quality that could be used to assess undergraduate education. The Task Force recommended that a specific Academic Senate committee be charged to work in collaboration with the VP-UE to establish such guidelines and measures.
Both the CEP and the GC submitted formal responses to the Academic Senate Chair concerning the report of the Academic Program Review Task Force (Letter from CEP Chair; Letter from GC Chair). Both committees concurred that the CEP should play an analogous role in program review for undergraduate education as the GC plays for graduate education and should work with the VP-UE and the Dean of the Graduate Division respectively to oversee the review process. Moreover, both committees agreed on the importance of conducting combined reviews of graduate and undergraduate programs housed within departments and of developing relevant criteria and associated metrics, particularly in the area of undergraduate education, to assess these respective programs. Finally, they agreed that the GC and Graduate Division would share responsibility for reviewing graduate groups that exist outside of departments and the CEP and VP-UE would do the same for review of undergraduate programs housed outside of departments.
The campus is committed to creating a program review process that includes undergraduate education. At the time of writing, the Chair of the Academic Senate has forwarded the official Academic Senate response to the Task Force report to EVCP Paul Gray. It recommends that the EVCP charge a working group to move forward with the implementation. This working group would draft a revised "Comprehensive Guide for the Review of Existing Instructional Programs & Units," guided by the principles outlined by the Academic Senate. It is expected that at least one departmental review with a fully developed undergraduate education component will take place during 2002-03. As part of that pilot review, we plan to develop an assessment mechanism to evaluate how well the new review process is functioning.
© Copyright 2002 The Regents of the University of California at Berkeley. All Rights Reserved.
To report problems with this website, please contact the webmaster.