Two theories of Assessment

by Shirlee

As assessment has become ever-more-prominent in education and the non-profit world -- a trend that has been building for the past 25 years, according to my reading -- two distinct sets of reasons for assessing have been given. Often they live side-by-side in uneasy alliance. But they are very different, and that difference could make a BIG difference for faculty lives.

Assessment Theory #1 -- ACCOUNTABILITY

The problem assessment is supposed to solve:
Lots of money goes into education and non-profits, and it is hard to tell if it is well-spent or wasted.

The thinking: Lots of money, for example, is given to feed the hungry. Is it being spent well? Are there fewer hungry people than there would have been without the spending? Could it have been spent more effectively? It is hard to say.... Education is in this same boat -- lots of money gets spent, but how can we tell if it is being spent effectively? In this thinking, this aspect of the non-profit/education world contrasts sharply with commercial enterprises. That is, in a market environment, there is a straightforward way to tell if an investment was a good one from a business point of view -- did it lead to the creation and selling of a good or service that enough people wanted to buy so as to make a profit? In a market, it is possible to compare investments over time, using the metric of return on investment, to judge whether or not money was well-spent. But in education, as in most of the not-for-profit world, it is harder to tell what is "working" and nearly impossible to tell if one kind of investment in education -- for example, college scholarships to kids from poorer families -- yields a better or worse return on investment than another strategy -- for example, full-day pre-kindergarten for kids based on income eligibility.

In this context, assessment is intended to provide metrics that enable valid comparison of one possible use of funds with other possible uses. In the world of higher education, this leads to the call for standardized exams of basic skills -- usually writing and critical thinking -- to be given to all students (or a representative sample of all students), across all schools. The scores on these standardized tests would allow easy and clear comparisons of one institution with another. Indeed, there are several exams that are competing to fulfill this dream, such as the CLA (or for community colleges, CCLA) or the ACT CAPP. Lots of colleges and universities have responded to the call for assessment using this theory of assessment by mandating the use of one of these tests.... After the admistration gets the results, they then inform faculty of how well (or badly) they are doing. Poor results lead to lots of administration pressure on under-performing faculty.

Assessment Theory #2 -- CONTINUAL IMPROVEMENT

The problem assessment is supposed to solve:
Our world is in dire need of the skills and competencies characteristic of educated people -- primarily communication, collaboration, and critical thinking skills -- and this mission is so important, we must devise ways to identify and quickly roll-out best practices.

The thinking: teachers tend to be intrinsically motivated, curious innovators. However, they have historically worked in great isolation. One teacher might informally track what works and doesn't work in his/her classroom -- and make changes based on observations -- but this is primarily a matter of independent practice. Education is so important in our changed world, though, we have to take these kinds of individual practices and formalize them. In this way, assessments allow us to formally track the results of innovations, and more quickly and easily isolate components of the most effective education practices, so they can be shared more widely.

In this context, assessments must be in the control of faculty members, as it is their familiarity with their student population, curriculum, content, and current practices that leads to the determination of what to assess. Standardized assessments for a general population of students will not help (for example) history faculty to discern best practices for critical thinking in history. Faculty must control the assessment process, because they have the detailed familiarity to know how to focus an assessment.


At PCC, we have been firmly within the fold of Assessment Theory #2. It is, after all, what our accreditors have asked of us -- evidence that assessment of student learning is being used to improve both teaching and learning. It is also the process that is most respectful of faculty, so it is not suprising that a faculty Council would come up with this sort of a recommendation. Additionally, it is the only approach that could lead to results that would actually be useful to instructor practice.

Notice, however, that Theory #2 leads to ever more customized and distinct assessments, while Theory #1 leads to ever more generalized and standardized assessments. These two theories lead to incompatible pictures of what GOOD ASSESSMENT looks like. The more locally useful a particular assessment is to a given SAC, the less useful it will be to compare one college to another....

I mention all this because one of my heroines in the Assessment World is Trudy W. Banta, editor of Assessment Update: Progress, Trends, and Practices in Higher Education. In the most recent edition (Sept-Oct 2011) she has written a piece warning that "... the promise of assessment for improvement might be diminished by increased focus on assessment for accountability."

She writes:

...[A] significant portion of US colleges and universities may be moving in the direction of providing to the public information based on scores of standardized tests of generic skills that inevitably will be used to compare quality of institutions. It just seems to be human nature to hone in on those easy numbers when we seek a standard for making comparisons."

I offer two thoughts in this context:
(1) We, here at PCC, are lucky to have both an administration AND an accrediting agency operating from Assessment Theory #2. This is the approach that fully respects faculty as THE key players in driving program improvement. Although assessment takes up hours and energy, when done by faculty (and done well) it leads to results that make a difference in student lives -- it leads to continual program improvement and better learning outcomes.

(2) While we here at PCC are getting better and more proficent at assessing our programs and disciplines -- witness the amazing variety and innovation of assessment approaches across our SACs, with ever-better strategies and instruments -- it simply may not be enough to stave off the rush to standardization. At this point, there is still a national push for assessments that fit the picture from Theory #1. Assessing to compare one institution with another is very, very, very different from assessing to be able to do our important job ever better. While both are part of the rise of assessment, they lead to very,very,very different pictures of what good assessment looks like. A one-size-fits-all test looks ridiculous on one model, and the only thing that will work on the other....


Wait. I really meant:
very, very, very, very different.....

I'll keep an eye out and let you know what I see on the standardization horizon. Until then, we will continue with our PCC plan of asking for a splendid locally-controlled profusion of SAC-specific assessment!!