Tuesday, November 2, 2010

Phil Seder's Assessment Angst

I come from the world of business marketing. In that world, we plan, we produce and we measure success in order to refine programs and improve outcomes. I believe this to be a valid approach. I teach my students that it is an essential approach in the business world.

I thus find myself perplexed by my internal resistance to the idea of assessment that is being pushed down from the highest levels of government and has now come to roost at the community college level. There is no doubt in my mind that we can improve delivery and there is no doubt in my mind that assessment can lead to refinements in course content or delivery that results in better outcomes. So why my angst?

I think it is because in the business world, there is a fundamental difference from the world of classroom education (well, a bunch actually, but I’m going to deal with just one here). In business, marketers work with researchers to determine consumer or business needs. Those marketers pass requirements on to designers and engineers who develop the products or services. The engineers pass the designs to manufacturing and sales to create and deliver the products. And finally, the marketers (often with the aid of accountants or analyst) assess the results before another round of the same is initiated.

Now let's look at who performs these tasks in the education world. Basic research and determination of customer needs? At the classroom level at least, the teacher. Product design? The teacher. Product development? The teacher. Product delivery? The teacher. Product assessment? The teacher.

This is not to say that administrators do nothing. There are monumental issues of overall program development (e.g. Should we offer a nursing program), new program implementation, facilities construction and maintenance, marketing, budgeting, negotiation, discipline and, ah yes, even employee supervision. I thank my stars that trained professionals stand ready to perform these essential duties.

But the bottom line is that the classroom teacher is responsible for every aspect of the development and delivery of the product in the classroom. In other words, they perform tasks that are the responsibly of numerous differently trained professionals in the business world. I know this because I've managed products in both worlds and, frankly, it is one of the things that excites me about the daily challenge of teaching.

But therein lies a conundrum. To teach better, we are being told, we need to assess more. We need to be more like the measurement driven business world. The teacher though, is not like the business professional. They are responsible for the entire life cycle of product development and delivery in the classroom. To assess more means that they either have to 1) work more or 2) spend less time on planning, development and delivery.

Now we all know that there are those who can work more. But for many teachers I see around me, their waking hours during the academic year are filled with the activities of course development and delivery. I doubt if many would look kindly at giving up the few moments of personal time they have during the week. As to the old saw "work smarter, not harder," well, it's a trite and meaningless age-old piece of business wisdom, custom canned for delivery to the survivors of corporate layoffs as they contemplate a future of 24/7 employment. At the very least, when I have been in those situations, I had the comfort of higher compensation to balance the loss of personal and family time.

Today's reality of an assessment-driven education system though, is that the classroom teacher will have to cut back on planning and delivery activities to respond to the assessment demands. And the more dedicated the teacher, the more they will need to cut back, since they are the ones who already spend their entire waking time engaged in better delivery. The result: we know what we're getting, but what we know we're getting is worse than what we were producing before when we didn't know what we were getting. Read that twice, if need be, then, especially if you have been teaching for a decade or more, contemplate whether this does not seem to precisely speak to the ultimate results of modern educational policy at the high school level.

I see it in my own work. Even in the past five years, I have seen a creeping slide towards more meetings, more time discussing the nuances of outcomes language (e.g. our graduates don't communicate with coworkers, they interact with coworkers), and more time discussing assessment. Where I have cut back was first, in my personal life where I spend limited time on my once thriving second life as a sculptor, and second, in my course preparation and delivery. It's not like I drop professionalism; it's just that I make one course improvement per term where previously I made three or four or five. Or the extensive paper markups that I used to do, and for which I was often thanked by more serious students, have become rare and often replaced by a mere letter grade. It's not what I want. But the fact is the time spent responding to federal officials and accreditation committees must come from somewhere.

Ultimately, the question that I can imagine being thrown back at me is this: "Then would you just say to give up on assessment, even knowing that better assessment can create a better product?" in a nutshell, yes. Taking this back to my original business analogy, if a shortage of resources forced me to make a choice between building a better product or measuring consumer acceptance of that product, I would err on the side of building the best product I could. In a world of unlimited resources, I would do it all. In our real world of limited resources, trade offs will be made.

Of course, I do assess. In many ways and on many dimensions. I assess writing skills in business classes (which doesn't make one very popular, I might add), I assess math skills, I force students to present, I make them work in teams. Do I know scientifically whether my assessments are creating the best students, or whether I'm doing a better job teaching than I was five years ago? No. Intuitively I believe so. But I know I am putting the most I can into my product and I am comfortable that in a world of scarce resources, I am allocating my time in an ethical and professional manner.


  1. While I share the angst and recognize the overwhelming impact on a classroom instructor, I question why your ultimate answer "in a nutshell" is, yes. Why can't departments, managers, or in general, groups and systems have assessments that don't impact the classroom instructor so negatively? I do agree that this is the case currently, but if good assessment does lead to a better outcome, and the business model you reference has a wider group of professionals as part of the process, why not in education?

    What role if any, are the unions playing in this process? How are instructors perhaps limiting a valuable resource? How are administrators hindering this development? How must the systems within the organization limiting this growth potential?

    So, my answer in a nutshell is, no. Changes need to be implemented. And for the record, I'm glad you evaluate writing skills in business class.

  2. I'm SO glad you evaluate writing skills in business class. How cool would it be to have integrated classes that combine what a writing teacher (that's me!) and business class teacher are covering? But that's a whole other topic.

    I also so appreciate the clarity and sincerity of your ideas, Phil. I second, however, that if change is coming (necessary or not), then the next step is to understand in what way it can be implemented that is most efficient in using administrators' and teachers' time. What this plan of efficiency will be, of course, is the hard question that you are beginning to ask in what you have written, I believe.

  3. I appreciate Phil's post enormously.

    I've been struck by the willingness of some faculty to take seriously an assessment process that fails to understand that our work--our real work--will not fit into little boxes. I don't mean that to sound superior. I just mean it literally: what we do can not be assessed via the processes we are directed to complete. In addition, as Phil points out, the task of continuous assessment is simply deleterious in real ways to our lives and our teaching. It takes time, and any faculty member who has excess time on their hands is a mystery to me. Perhaps assessment is inevitable and inescapable, as the previous comments say. But can we at least be in some solidarity that its premise is set forward by people who may not understand what really takes place in a classroom? Can they not be gently told to do the assessment work themselves if it interests them so? I am deeply uneasy about top down directives imposed on teachers (anybody remember the cultural revolution in China?).

    Is everything really as inevitable as it looks?

  4. Dallas.

    Like you, I don't like wasting my time complying with top-down directives coming from people who do not understand the nature and value of the work I do.

    And therein lies the beauty of the assessment approach PCC is taking. There is no process faculty are directed to complete.. Instead, we are directed simply to complete SOME assessment process. The details are left up to you (and your SAC.) The thinking is that if faculty are in charge of the design and implementation of the assessment of their own learning outcomes, then the work can be crafted to meet real needs, answer real questions, and serve real purposes.

    The idea you express -- that our real work as educators does not fit into boxes -- is at the heart of this approach. While we are all teaching, we are teaching very very very different things. That diversity is vital. We must all assess (according to the directive), but our assessments can be as diverse as our subject areas and interests.

    It would be possible to hire in some assessment professionals, and ask them (on behalf of the administration) to assess the efficacy of the instructors. Lots of colleges and universities are actually doing just that. We, as faculty, would not have the extra tasks landing in our laps. That is an advantage of the outsourcing approach. But I personally think that approach has lots more little-box danger than the PCC way. (Though I appreciate your wish to tell administrators GENTLY to go hire out the work to their own assessment experts.) If administrators decide what to assess, who will assess it, how they will assess it, and what to do with the assessment results.... well, then I fear, we could see some big time time top-down directives.
    PCC has to respond to the demand from NWCC to assess. NWCC has to respond to the Dept of Ed demand to assess. The Dept of Ed has to respond to global changes in the politics and economics of our new world order. (Already, international students are picking institutions of higher ed with portable, transparent, and more credible credentials than those in the U.S.)

    If change is inevitable (which I think it is) I prefer being in control of that part of the process I can control... and that's why I like PCC's way of responding to this global change.