Measuring What We Value
Mason Explores Performance Evaluation Using Outcome Measurements
By Carrie Secondo
George Mason is at the forefront of developing new performance measures for higher education in Virginia and could soon be one of two state schools to participate in a performance-based funding pilot program.
President Alan Merten says by moving away from traditional input/ output measurements toward an outcome-based model, George Mason can measure what it values, rather than placing value on what is measured in traditional performance evaluations.
"If you don't measure what you value, it won't improve," Merten says.
Currently in Virginia, performance is measured based on easily collected data, including student graduation and retention rates; the percentage
of budgets allocated to instruction; credit hours generated by full-time instructional faculty; research and public service expenditures per full-time faculty; classroom space use; and job placement after graduation.
Student and alumni satisfaction are examples of outcome performance measures.
Outcome measures, unlike input/output methods, focus more
on student satisfaction and quality
of education. These outcomes can be evaluated based on survey results, rather than database information.
"When state agencies look for performance measures, they typically look for what they can grab off a database, and for the most part those are input oriented," says Senior Vice President Maurice Scherrens. "Occasionally they have an output performance measure like graduation rates, but they never get outcome performance measures, and that's what we think we ought to be moving toward."
Graduation rates are a classic example of how traditional output measures can leave a lot of valuable information out. "If we looked only at graduation rates, we would know a student graduated, but we wouldn't know if she went into the field she studied, or if she did well in whatever field she pursued after graduation," Scherrens says.
Traditional methods of evaluation don't tell George Mason what it needs to know based on what it values.
"We want to know if a George Mason graduate is satisfactorily pursing a career; are they satisfied with their career choice; and are they qualified to advance in itÑthose are the outcome measurements we value," Scherrens says.
Another example of how outcome measurements could better evaluate how well George Mason is doing would involve modifying traditional job placement rates to include employment rates.
"Placement rates are less important to us than employment rates, because many of our students are already employed when they come to George Mason. So the question is were they, after graduation, employed in a upwardly mobile position or have they enhanced their chances for promotion," Scherrens says.
Already George Mason's efforts to measure outcome have been used to improve its quality of education. In response to results on a student satisfaction survey conducted by the Office of Institutional Assessment last year, the College of Arts and Sciences hired more faculty to reduce the number of students in some of its largest classes.
"This is one of the first times freshman classes have been given faculty to reduce the class size, and that was a result of the student satisfaction survey," Scherrens says.
Another traditional measure uses the percentage of classes taught by full-time faculty to assess quality of instruction, without taking into consideration the expertise of adjunct faculty. Using an outcome-based approach, George Mason can better assess the quality of instruction by evaluating whether the student is satisfied with a class taught by an adjunct.
"Some of our adjunct faculty have strengths equal to or greater than our full-time faculty, because of where we are located. For the most part, our adjuncts are outstanding practitioners, so once the student is in the class they think it's the greatest class in the world," Scherrens says.
Merten, along with Scherrens and state officials, has been developing new performance measures that could be used for higher education funding in the near future. So far they have defined 12 performance measures but have yet to set exact criteria, because measurements would be unique to each school. Reaching a performance agreement will be much like getting a new degree approved. "There will be a lot of conversation among the president, deans, directors, and the BOV before a proposal is sent to the state level for approval," Scherrens says.
In 1998, former governor George Allen proposed performance-based funding for Virginia's colleges. Under his plan, funds awarded based on performance would have been earmarked for the institutions' highest priorities. The funding model under consideration now allows universities to use the money as they deem appropriate.
Along with Virginia Tech, George Mason has asked to be the first to pilot a performance-based funding model, if the Virginia General Assembly approves such a program.
