Response to AB 94: Goal Setting for Academic Performance Measures for
Academic Sustainability Plan
RESOLVED: That the Academic Senate of the California State University (ASCSU) express its concern about the academic performance measures mandated by AB 94 and included in the November 2014 California State University (CSU) Academic Sustainability Plan; and be it further
RESOLVED: That the ASCSU request that the March report, in which the CSU is required to submit documentation on the 16 performance measures, include concerns on the specific measures used; and be it further
RESOLVED: That the ASCSU asserts that the AB 94 performance measures are problematic for the following reasons:
1. It is unclear how the performance measures combine to provide a unified, overall measure of the quality of education provided by the CSU.
2. Given the highly diverse universities in the CSU, the global nature of the performance measures makes it difficult to meaningfully disaggregate the measures to a campus level.
3. The overall focus on outcomes measures and degree production prevents a more nuanced picture of the measurable ways in which the CSU meets its mission of access, affordability and quality.
4. The validity of many of the performance measures is questionable given the variety of factors that may affect the variables involved.
5. The use of the AB 94 performance measures to make budgetary decisions may have consequences for the mission of the CSU that need to be fully explored.
6. Other measurement issues need more attention, including the use of absolute values for goals, how campuses will be benchmarked, the implications for campuses of missing a quantitative goal, and what data will be used for the baseline comparison for each measure.
7. The budgetary consequences of the performance measures need further elaboration
; and be it further
RESOLVED: That the ASCSU strongly recommend that the CSU establish a task force composed of equal numbers of faculty, selected by the ASCSU Executive Committee and administrators to review the utility and larger implications of the performance measures in AB 94, and propose alternatives, as needed, to the Legislature and Department of Finance; and be it further
RESOLVED: That the ASCSU recommend that the deliberations of the performance measures task force included but not be limited by the following questions:
1. What are the long-term implications of the AB 94 performance measures for the mission of the CSU? In an era of reduced resources, by what criteria should the CSU prioritize the goals established as a result of the measures?
2. What is the best way to achieve a balance between the Legislature’s desire for performance measures that assist in the “effective and efficient use of available resources” (AB 94) and performance measures that present a comprehensive picture of the CSU’s three-fold mission of access, affordability and quality?
3. What is the appropriate balance between “input” and “output” measures to use to present the most accurate picture of academic quality in the CSU?
4. What is the best way to account for factors that may influence the measurement of performance variables?
5. What are the best measures to use to account for the diversity of CSU universities?
What is the estimate of the total dollars that will be allocated based upon performance funding? Will it be the entire general fund allocation to the CSU? A percentage? A percentage per goal? And given the complexity of multiple performance indicators, what are the preliminary decision rules for allocation of funds across campuses?
; and be it further
RESOLVED: That the ASCSU distribute this resolution to CSU Board of Trustees, CSU Chancellor, CSU campus Presidents, CSU campus Senate Chairs, CSU Provosts/Vice Presidents of Academic Affairs, California State Student Association Board of Directors, Academic Senate of the University of California Executive Committee, California Community Colleges’ Board of Governors, Assemblyman Lui, Legislative Leadership, California Community Colleges Academic Senate Executive Committee, Director of Finance, Chair - Senate and Assembly Budget Committees, Chair - Senate and Assembly Budget Subcommittees on Higher Education, Chair - Senate and Assembly Appropriations Committees - Chair, Policy Committees with Jurisdiction Over CSU, Chair - Assembly Committee on Higher Education, and the Chair - Senate Standing Committee on Education.
California’s Budget Act of 2014 requires the CSU to produce an academic sustainability plan for the 2015-16, 2016-17 and 2017-18 fiscal years. As part of that plan, the CSU is required to report on 16 performance measures meant to “inform budget and policy decisions and promote the effective and efficient use of available resources.” (AB 94) The plan is due to the legislature and Department of Finance in November 2014. By March 1 of each year, the CSU is required to report on progress made in achieving goals based upon the performances measures. The ASCSU has concerns about the performance measures contained in the Budget Act of 2014, and is recommending that a joint faculty-administration task force be created to review the utility of the performance measures and to recommend alternatives, as needed, to the Legislature and Department of Finance.
The Budget Act of 2014 explicitly states that the current Administration in Sacramento is moving away from funding higher education based upon enrollment targets, to a model that is based upon increasing productivity and improvement in outcomes, including affordability, timely completion rates and quality programs. The shift to outcomes, to the exclusion of some traditional “input” measures, paints a blunt picture of the CSU, rather than the more nuanced, comprehensive and accurate portrayal a mixture of the two types of measures would provide. And, when taken collectively, the performance measures focus almost exclusively upon degree production, to the exclusion of measuring the quality of programs provided by the CSU. In addition it is possible that the use of the performance measures as criteria by which to make budgetary decisions will force the CSU to make decisions that will fundamentally alter its mission. Given the diversity of universities in the CSU, it is difficult to meaningfully disaggregate the performance measures to the campus level. And finally, the validity of many of the performance indicators is difficult to establish given the variety of factors that may affect measurement.
Several of the performance measures included in AB 94 may be of use. Measures of numbers of transfer students and Pell Grant students are rough indicators of access; and, time-to-degree measures and graduation rates are indicative of student progress and achievement, one aspect of academic quality. However, depending upon whether the number of Pell Grant students is a positive or negative indicator of success, it may be an incentive for campuses to target their recruitment efforts solely on low income students. Conversely, it may be an incentive reduces the number of low income students. And, a measure such as expenditure per degree is virtually meaningless in measuring academic quality. The cost of a degree varies by the nature of the program, whether the degree is an undergraduate or graduate degree, the infrastructure necessary for a given program, and the like. For similar reasons, cost can also vary by specific course. In addition, if the expenditure per degree goes up, it is unclear whether that would be considered a measure of increased quality or increased inefficiency.
Another example of a problematic measure of academic quality is the average number of course credits earned at graduation (disaggregated by freshmen, transfers). The CSU has already reduced completion requirements to 120 semester units for most majors. And while the number of course credits earned can be another measure of potential timely progress to degree, the danger is reducing the definition of quality an arbitrary maximum, rather than optimum, number of units, and focusing on a numerical representation of a degree, rather than its substantive content (such as General Education and Major components, intern/service learning options, co-curricular opportunities for students, and so on).
First year units earned is an example of a measure that is affected by other factors that call into question its validity. The more remediation that is required, the fewer the actual first year credits that will be accumulated, which would result in a smaller proportion of students who will achieve a timely progress measure. As with the number of Pell Grant students, this could create a potentially perverse incentive for a campus to disinvest in marginal students (or recruit students with a higher probability of success).
In addition to the above, the goals established according to the performance measures may be in competition with one another. As pointed out by the CSU in its draft sustainability plan, it may be difficult to accomplish multiple goals if they pull the institution in different directions and make competing demands upon resources. The primary example of this is the tension between measures geared toward access (enrollment measures) and retention (graduation rates for new freshmen, transfers). An effective way to increase graduation rates is to restrict access. Restricting access would alter one of the most socially significant aspects of the mission of the CSU, and potentially harm the diversity of the institution.
And finally, the ASCSU urges the task force to explore in more depth the budgetary implications of the performance measures. The amount of dollars allocated by the state to the CSU based upon the performance measures is unclear. And if the CSU is expected to allocate funds internally based upon individual campus performance, a wholesale re-evaluation of the current criteria for funding is needed. This is especially important given the complexity of multiple performance indicators.
AB 94 full text available at: http://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201320140AB94
California Budget Act of 2014: Higher Education available at: http://www.ebudget.ca.gov/2014-15/pdf/BudgetSummary/HigherEducation.pdf
CSU Draft Academic Sustainability Plan
AB 94 Performance Measures
Article 10 (commencing with Section 89290) is added to Chapter 2 of Part 55 of Division 8 of Title 3 of the Education Code, to read:
Article 10.Expenditures for Undergraduate and Graduate Instruction and Research Activities
(a) The California State University shall report biennially to the Legislature and the Department of Finance, on or before October 1, 2014, and on or before October 1 of each even-numbered year thereafter, on the total costs of education at the California State University.
(b) The report prepared under this section shall identify the costs of undergraduate education, graduate academic education, graduate professional education, and research activities. All four categories listed in this subdivision shall be reported in total and disaggregated separately by health sciences disciplines, disciplines included in paragraph (10) of subdivision (b) of Section 89295, and all other disciplines. The university shall also separately report on the cost of education for post-baccalaureate teacher education programs. For purposes of this report, research for which a student earns credit toward his or her degree program shall be identified as undergraduate education or graduate education, as appropriate.
(c) The costs shall also be reported by fund source, including all of the following:
1. State General Fund.
2. System-wide tuition and fees.
3. Nonresident tuition and fees and other student fees.
(d) For any report submitted under this section before January 1, 2017, the costs shall, at a minimum, be reported on a system-wide basis. For any report submitted under this section on or after January 1, 2017, the costs shall be reported on both a system-wide and campus-by-campus basis.
(e) A report to be submitted pursuant to this section shall be submitted in compliance with Section 9795 of the Government Code.
(f) Pursuant to Section 10231.5 of the Government Code, the requirement for submitting a report under this section shall be inoperative on January 1, 2021, pursuant to Section 10231.5 of the Government Code.
Article 10.5 (commencing with Section 89295) is added to Chapter 2 of Part 55 of Division 8 of Title 3 of the Education Code, to read:
Article 10.5.Reporting of Performance Measures
(a) For purposes of this section, the following terms are defined as follows:
1. The “four-year graduation rate” means the percentage of a cohort that entered the university as freshmen that successfully graduated within four years.
2. The “six-year graduation rate” means the percentage of a cohort that entered the university as freshmen that successfully graduated within six years.
3. The “two-year transfer graduation rate” means the percentage of a cohort that entered the university as junior-level transfer students from the California Community Colleges that successfully graduated within two years.
4. The “three-year transfer graduation rate” means the percentage of a cohort that entered the university as junior-level transfer students from the California Community Colleges that successfully graduated within three years.
5. “Low-income students” means students who receive a Pell Grant at any time during their matriculation at the institution.
(b) Commencing with the 2013–14 academic year, the California State University shall report, by March 1 of each year, on the following performance measures for the preceding academic year, to inform budget and policy decisions and promote the effective and efficient use of available resources:
1. The number of transfer students enrolled annually from the California Community Colleges, and the percentage of transfer students as a proportion of the total undergraduate student population.
2. The number of low-income students enrolled annually and the percentage of low-income students as a proportion of the total student population.
3. The system-wide four-year and six-year graduation rates for each cohort of students and, separately, for low-income students.
4. The system-wide two-year and three-year transfer graduation rates for each cohort of students and, separately, for each cohort of low-income students.
The number of degree completions annually, in total and for the following categories:
A. Freshman entrants.
B. Transfer students.
C. Graduate students.
D. Low-income students.
6. The percentage of first-year undergraduates who have earned sufficient course credits by the end of their first year of enrollment to indicate they will complete a degree in four years.
7. For all students, the total amount of funds received from all sources identified in subdivision (c) of Section 89290 for the year, divided by the number of degrees awarded that same year.
8. For undergraduate students, the total amount of funds received from all sources identified in subdivision (c) of Section 89290 for the year expended for undergraduate education, divided by the number of undergraduate degrees awarded that same year.
The average number of course credits accumulated by students at the time they complete their degrees, disaggregated by freshman entrants and transfers.
A. The number of degree completions in science, technology, engineering, and mathematics (STEM) fields, disaggregated by undergraduate students, graduate students, and low-income students.
For purposes of subparagraph (A), “STEM fields” include, but are not necessarily limited to, all of the following: computer and information sciences, engineering and engineering technologies, biological and biomedical sciences, mathematics and statistics, physical sciences, and science technologies.
Approved Unanimously – November 7, 2014