4(a). Institutional Uses of Data
1.2 (indicators and evidence), 4.1 (data linked to decision-making), 4.3 (data informs planning), 4.4 (data collection for improvement), 4.5 (strategic data needs), 4.6 (assessments used), Question 3(data collected and used effectively) under Strategic Thinking and Planning (4), Question 6 (culture of evidence) under Commitment to Learning and Improvement (4)
The collection, dissemination, and use of Institutional Research data in decision-making are critical to improving educational effectiveness. The management of institutional data is decentralized at Berkeley. This permits decision-makers to use data effectively in their planning processes, but also creates a number of challenges. For example, the campus has not explored in depth the possible development of a campus-wide strategy for assessing student learning outcomes. This essay discusses the challenges posed by decentralization and efforts underway to integrate institutional data systems, and proposes some ways to address the need to assess comprehensively student learning outcomes.
Institutional Data Management in a Decentralized Environment
Institutional research and planning at Berkeley take place in a variety of venues and not simply, as at many other universities, in a centralized institutional research office. Decentralization of institutional data ensures that decision makers throughout the campus have efficient and reliable access to the data and the analyses that they require.
Six offices provide most of the institutional data to the Berkeley campus: Planning and Analysis, Student Research, Admissions and Enrollment, Graduate Division, Space Management and Capital Projects, and Sponsored Projects. Several of the larger colleges and schools also employ analytical staff. Managers and analysts from the institutional units that provide data also serve on standing or ad hoc campus committees. Their primary function is to provide appropriate data to support committee decisions. Their participation ensures that committee members fully understand complex statistical information and allows campus-wide as well as departmental decisions to be "data driven."
For example, in the past year the Strategic Planning Committee (SPC) and Near Term Planning Subcommittee requested from the Office of Planning and Analysis a number of special data analyses to identify departments where faculty workload and student demand for a major were highest. Similarly, a special report prepared by Admissions and Enrollment on the academic success of athletes at Berkeley for the Admissions, Enrollment, and Preparatory Education Committee and the Undergraduate Admissions Coordination Board will lead to a more systematic review of athletic admissions by faculty, especially in the admission of high risk athletes. At the request of the Affordability Steering Committee, the OSR conducted surveys to evaluate the impact of a variety of costs of attending Berkeley on freshmen and transfer student populations. In many cases, data provided for a single committee's use may be shared with other decision-makers or committees as appropriate.
Challenges of Decentralization
Having multiple campus units perform institutional research functions creates a number of challenges, including content and technical issues, integration of systems and data, and issues of access and ownership. Some of the pitfalls of decentralization are that (a) institutional data are at times not shared effectively with planners and decision-makers, with the result that planning efforts are not fully informed; (b) institutional data elements are not fully shared among data providers or analysts across campus; (c) efforts may be duplicated as several units collect and analyze similar data; and (d) data generated for reports to departments or campus committees are often provided on a piecemeal basis, designed to answer a specific question or set of questions rather than being part of a broader evaluation or research agenda.
As are most large institutions nationally, the campus is now engaged in the transition to relational database and web-based technologies which, in effect, combine the advantages of the centralized and distributed models of computing. This transition requires significant changes in business policies and practices that ultimately will increase efficiency and accountability by minimizing redundancy and improving data quality. Among the many technical and policy issues raised by the new technologies, two seem particularly germane to the evolving organization of institutional data collection and analysis at Berkeley: (a) how to make data accessible while ensuring security and confidentiality, and (b) whether the present quantity and configuration of analytical and technical staff remain optimal. Both issues are the subjects of continuing discussion and evaluation.
Efforts Toward Data Integration
Recognizing that data integration is an important requisite for improving the campus's organizational and operational effectiveness, the Chancellor launched a Data Integration Initiative to standardize data and data access policies across campus, and to improve access to campus information through the use of Web-enabled technologies and on-line analytical processing tools. The Working Group's May 2002 report on data integration included five major recommendations (Data Integration Workgroup):
The Data Stewardship Council will also address technical and systems issues including the development of policies related to data access and security, a student data warehouse, and campus data dictionaries.
Establishment of a Data Gateway
The Institutional Data Gateway, established in Spring 2002, provides the campus community with instantaneous access to basic institutional data and links to UC system and national data. The following five campus databases are included in the Data Gateway:
Cal Profiles, established in November 1998, is a comprehensive, longitudinal view of more than 400 data elements for all campus units, which can be viewed at any level from the campus down to a specific unit. The Office of Planning and Analysis provides hands-on training and an on-line guide for campus staff.
Cal Profiles Plus, created in November 2001, accesses the same database as Cal Profiles, but provides more detail and allows the user to drill down and create charts.
The Common Data Set (CDS) is a collaborative effort between publishers and educational institutions to standardize the set of annual data elements requested for educational publications. The campus web site was launched in August 2000.
The Performance Metrics web site, on line since September 2001, provides trend data on campus metrics used to assess the degree to which organizational goals are being met. It also provides a one-year snapshot comparing Berkeley to a group of 12 peer institutions—5 privates, 5 publics, and 2 UC campuses. Comparative data is available on a series of measures.
The Student Data site maintained by the OSR allows users to select and view summary data on applicants, registered undergraduate students, and undergraduate degree granted.
Student Learning Outcomes
Building on the Data Integration Initiative, as part of its Educational Effectiveness Review, the campus will explore the possible development of a campus-wide strategy for assessing student learning outcomes. The following steps have been identified as ways the campus can improve assessment of student learning:
© Copyright 2002 The Regents of the University of California at Berkeley. All Rights Reserved.
To report problems with this website, please contact the webmaster.