want some money?

by Shirlee

The Learning Assessment Council now has a budget!!

This is our 4th year of existence, and up until now our mission has been a bit temporary and provisional: we were helping PCC feel its way forward into the new era of "accountability" in higher ed. We insisted that assessment be respectful to both teachers and students, and that the efforts should not be wasted -- but should lead to meaningful and practical results that could actually be used to improve teaching and learning.... This process meant that faculty had to be deeply involved in designing assessments, as well as implementing them -- since it is faculty who will be most directly involved in implementing any program or instructional changes suggested by assessment results.

Our recommendation that assessment be faculty-owned and faculty-driven has, indeed, resulted in more work for faculty members -- which was not a universally popular consequence of our advice. But this fall. talking with people who had been through one or two of the assessment cycles, it was clear that opinion was changing. Enough people had learned enough interesting things from their program or discipline assessments, and were willing to talk about them, to change the energy. And then our work was recognized by our accrediting agency, NWCCU, which wrote that we were substantially in compliance with their standard on assessment. Hurray for us!!

And, to top it off, the council got our very own budget. Not a lot (who gets what they really need these days?) But not nothing, neither. So, in that very first meeting to discuss what to do with the $$, we all knew immediately what we had to do:

--offer it up to support the creative and exciting assessment activities being cooked up by PCC SAC members.

(OK, we did briefly consider taking the entire council to Hawaii, but we gave that up pretty fast....)

It would have been lovely to have this all ready as SACs were making their assessment plans in the fall. But life doesn't always offer up perfect timing... So we decided to offer it late, rather than not at all, on the assumption that people are never unhappy about getting more money than they anticipated.

The time line is tight, though. We are asking for applications by Feb 3. We have about $12,000 to give away, and we want to give it away in chunks up to $2000 to support SAC assessment projects. What might you use it for? Here are just a few ideas.

• To invest in an assessment instrument that has been developed specifically for your field.
• To hire an assessment expert or consultant to advise you on assessment design.
• To acquire some software that would make your assessment cycle more effective or efficient.
• To send a SAC member to a conference on assessment in your particular field.
• To hire a psychometrician to help interpret assessment results.

These are just a few ideas -- and not particularly creative ones -- but I am hoping you get the idea. PCC is just one player in the world of Higher Ed, as we are all seeking to move into "evidence-based educational practice." There is a lot going on out there, and some of it would perhaps be particularly helpful to you, if you just had a bit of money to access it.

The application documents are available both from the Learning Assessment Council page and the Staff Development page (intranet).

Assessment coaches will be contacting SAC chairs to see if they have any interest in filing an application, and to offer help. If you want to get more information, or float an idea, please call:

Shirlee Geiger chair of the Learning Assessment Council 971 722-4659, or
Michele Marden, vice-chair 971 722-4786

now tell me again....who are the good guys?

There has been a lot of attention to for-profit colleges in the news of late, with a story-line that plays especially well with the generally left-leaning audience of educators. It goes something like this:

For-profit colleges exist for profit and -- just like in the general world of capitalism and the self-interested (aka selfish and greedy) competition of the marketplace -- the people who run them are willing to use some rather suspicious tactics when going for that profit. For example, for-profit colleges charge huge tuitions, and use blatantly false claims when assuring students that highly paid employment after graduation means tuition debt makes sense. They inflate the rate of completion and graduation of their students. And then they inflate the rate of graduates working in their field, along with the money they make in those jobs. They lie -- in order to make a profit. And then they are unconcerned about the wreck they make of the lives of the students whose money they so callously take...As long as they make money, that is all that matters.

I have heard this narrative from lots of places, and it made sense to me. The implicit contrast, of course, is with the noble people who work in the not-for-profit world of higher ed -- willingly forgoing the higher pay of the private sector in order to pursue the calling of seeking knowledge for its own sake, and passing it on to the eager young minds waiting to be shaped and guided....That would be me and my colleagues at PCC.

Alas, I had this little vision of the good guys and bad guys of higher ed shaken up last year at the American Association of Community Colleges, when I went to a session put on by Peter P. Smith. I went to hear him only because of his bio. He had served as the president of a community college in Vermont, and then as the founding president of the California State University at Monterey Bay. But then he left the noble not-for-profit world of higher ed to join Kaplan ( !) as a senior vice-president. (Kaplan is the largest provider of for-profit educational services in the world at the moment.) This, it seemed to me, was a MAJOR act of disloyalty and betrayal. How could anyone do that!?? How could he live with himself?!

I don't know exactly what I expected when I went to hear him.... but whatever it was, it wasn't what I got. First you need to know that a lot of marketing goes on at the AACC. There is an entire cavernous hall of vendors shilling expensive products, in row after row after row of booths. Lots of glossy pieces of paper get distributed. Logos are everywhere. Signs of the money to be made in higher ed are ubiquitous. The pure nobility of the pursuit of truth gets a bit lost in the hustle. In this context, it is easy to get a bit cynical. But 5 minutes into the presentation by this turncoat betrayer of the not-for-profit nobility of education and it was clear to me.... this guy is a serious idealist. It sounded to me like he believes more deeply in the intrinsic value of education than the most starry-eyed philosopher of education I ever met. I was flabbergasted! My conceptual categories were all confused! My sense of who is who was turned upside down. I felt that kind of vertigo that comes from having basic beliefs challenged.....

It has taken me a while to digest what I heard from him. He has a blog if you want to go and read his thinking: Peter P.Smith (He also has a book, but I haven't read it yet -- Harnessing America's Wasted Talent.) This all came back to me when I ran into a short article in Inside Higher Ed that quoted from him extensively. I am going to boil everything down, and no doubt oversimplify this. But here is the message I get from him, in a nutshell.

• The world needs educated people now.
• A lot.
• The education techniques currently being used were good enough in previous eras (when we only needed an educated elite). They don't work now.
• The accountability movement is all about bringing education into the information age, and finding ways to meet the new demands:
• 100% of our citizens highly educated with skills in collaboration, communication, and critical thinking.
• The biggest obstacle to developing new and effective education to meet the changed demands on higher ed are professional educators who resist change, and use their organizations to resist change effectively.
• The for-profit education sector is the newest, and the forces resisting change are the least well organized there.
• SO the for-profit education sector can and will lead higher ed into identifying and recognizing effective education techniques.

In this scenario, the people who are personally profiting in the non-profit education world -- the teachers and advisers and admins and APs like you and me -- are the major impediment to education that works.

Here at PCC, the Learning Assessment Council has adopted a strategy that goes against the smart and idealistic claims of Peter P. Smith. We think that faculty and CC staff can serve to drive a change to more effective education, not just stand in the way. We have charged YOU with crafting assessment strategies to see how you can meet student needs ever better. Still, I can see why Peter P. Smith has placed his bets against us. People who have benefited from the ways things have been done for a long time are quite often the most resistant to changing them. This phenomenon can be observed in industries and organizations across all sectors and around the world, as we have all scrambled to catch up with the changes we have witnessed the last two decades.

Life is changing in Higher Ed.... Help shape how PCC responds to the new demands. Get active in your SAC's program/discipline assessment project! Our students -- and the world that needs their skills -- will be the beneficiaries....

Data IDs Best Practices

by Shirlee

Sometimes I try to describe the accountability movement in higher education in words that can fit into the proverbial nutshell. That's when I rely on an analogy with the medical profession. Here is how it goes:

• The practice of medicine has traditionally been considered a profession where the practitioners (the doctors) are considered to be "experts" who we are all asked to trust.
• As a result, there have been few ways for someone "shopping" for a doctor to meaningfully compare one M.D. with another.
• Even so, it is known that some doctors do, in fact, get better results -- and often at lower costs -- than other doctors, treating the same conditions in similar patients.
• The cost of medical care has skyrocketed of late, and the mechanism we have created for payment (work-based insurance) is leading to huge social disparities, with a clear consensus that something has to be done, even as there is no consensus over what that is.
• Rumblings have been going on for a while now that one way to contain costs and increase access is to assess patient outcomes, in order to identify BEST PRACTICES and then make that information available to patients and taxpayers.

In the above story, we can change all mention of doctors to professors, and patients to students, and everything works the same....Really.

• Teaching in a college or university has traditionally been considered a profession where the practitioners (the professors) are considered to be "experts" who outsiders have been asked to trust.
• As a result, there have been few ways for someone "shopping" for a college or teacher to meaningfully compare one option with another.
• Even so, it is known that some colleges and teachers do, in fact, get better results -- and often at lower costs -- than others, even when the student populations are very similar.
• The cost of higher education has skyrocketed of late, and the mechanism we have created for paying (student debt) is leading to huge social disparities, with a clear consensus that something has to be done, even as there is no consensus over what that is.
• Rumblings have been going on for a while now that one way to contain costs and increase access is to assess student outcomes, in order to identify BEST PRACTICES and then make that information available to students, their families, and taxpayers.

Just like practitioners in the medical field, those of us in Higher Ed are being asked to:

• expand access to our services
• get ever-better outcomes for those who enter our doors
• and do this with less money per student.

There are, however, points of dis-analogy between the two fields.

• There are usually fairly clear indicators of success or failure (like mortality rates)with medical procedures --but success is harder in Higher Ed. If someone takes some community college classes, doesn't get a degree, but does get a promotion at work, is that success? or is it failure?
• In the medical field there are some service-payers that are so large, and who have been keeping records for so long, that there is LOTS of data to be mined. The biggest and best of these data piles comes from Medicare and Medicaid -- but for higher ed, there is no comparable keeper-of-records who could furnish us with data to study. Instead, we are in the early stages, via assessment of learning outcomes, of gathering that data.

Now I mention all this because I read today that the HUGE pile of data on patient outcomes is about to be released, in a format that will make it especially search able. Here is the link, plus a short excerpt:
http://my.earthlink.net/article/hea?guid=20111205/3c47d411-d964-4fce-933a-d1d1111584f2

"The government announced Monday that Medicare will finally allow its extensive claims database to be used by employers, insurance companies and consumer groups to produce report cards on local doctors — and improve current ratings of hospitals.

"By analyzing masses of billing records, experts can glean such critical information as how often a doctor has performed a particular procedure and get a general sense of problems such as preventable complications.

"Doctors will be individually identifiable through the Medicare files, but personal data on their patients will remain confidential. Compiled in an easily understood format and released to the public, medical report cards could become a powerful tool for promoting quality care.

"There is tremendous variation in how well doctors do, and most of us as patients don't know that. We make our choices blind," said David Lansky, president of the Pacific Business Group on Health. "This is the beginning of a process to give us the information to make informed decisions." His nonprofit represents 50 large employers that provide coverage for more than 3 million people."

Notice that the ratings are happening on two levels -- the hospitals (analogous to the colleges) and the doctors (analogous to the instructors.) Many colleges have already taken steps to help create a data set that can be used to compare one institution to another, by using one of the standardized tests (usually of critical thinking and communication) that have been created to allow just such comparisons. Instead of that route, we here at PCC have asked SACs to create or adopt assessment instruments that can deliver info they need to continually improve instruction. This gives us locally useful information, but no way to compare ourselves, as a college, to others. But so far, neither approach (standardized test, customized SAC assessment) will provide a way to meaningfully compare one instructor to another, the way the Medicare info will allow comparisons of one doctor to another.... Still, I say, any data that is aggregated can be disaggregated. And I think it is wise to attend to trends in the medical world, as hints of what will be coming our way.

Some of all of this makes me joyful. The faster we can figure out -- and share around -- what works, the more our students will learn. According to an article Linda Gerber sent my way, there is now more student debt than credit card debt in the US of A. This is a staggering realization. Go read this and weep: http://www.usatoday.com/money/perfi/college/2010-09-10-student-loan-debt_N.htm
But some of all of this makes me wonder how many of the traditional ways of higher ed will be changed beyond recognition in this process. ...

Evidence-based educational practices are a new trend, just like evidence-based medical practices. When my oncologist, 4 years back, laid before me the success rates of various treatment options for my kind of cancer, and helped me poke through the list to decide what to do, I was very grateful for this trend. (Since this pre-dates PCC's insurance for part-time faculty, my insurance wasn't that great -- since it was an individual policy, I didn't get the advantage of group rates -- and cost was one of the factors I considered.) Will the day come when there is an analogous approach to selecting college or college teachers? -- a high school college adviser lays out the same kind of data on rates of learning for college writing or critical thinking, and compares what is available to the student's aspirations and budget?

And should such a day come, how will PCC look as an educational choice?

These are among the interesting questions of our times....

NWCCU pressures us.... and who pressures NWCCU?

by Shirlee

Assessing student learning outcomes through PCC's SACs was put in place in response to pressure from our accrediting agency, The Northwest Commission on Colleges and Universities. The NWCCU upped the pressure on us in 2010 by saying that PCC was not in compliance with their standard on assessment for on-going improvement of teaching and learning. Many people here at PCC then stepped up to the plate in 2010-11 to create and implement assessment plans and to file reports on what they learned. For many faculty members, it was the first time accreditation had ever been directly considered.

Even though the pressure on us has lessened a bit, you still might be interested in knowing that the six regional accrediting agencies are feeling some pressure of their own. There are three separate bodies that each take a chunk of oversight of higher ed -- (1) the Federal government, through the Dept of Education, (2) state governments, who write and enforce a wide variety of requirements and standards, often directed to technical or vocational education and (3) the five regional accrediting agencies, including NWCCU,who require a routine of self-study and then provide peer review. These three bodies have intertwining connections. For example, eligibility for financial aid from the Feds is tied to attendance at a college or university accredited by one of the accrediting agencies.

It is this financial connection that is one prominent focus of a new advisory committee to the Dept of Education. In their report, the committee explores the reasons for and against severing this link between accreditation and financial aid. One possible future route they outline is to allow states to monitor educational quality -- reducing the power and role of both the federal Dept of Education and the accrediting agencies. Another possible future route puts more power into the Dept of Ed, and weakens the state role and the importance of the accrediting bodies.

The report is only 11 pages long, so you might just give in a glance. To get to the report, please navigate to insidehighered.com and then click on the hot line labeled "discussion draft." What gets decided is ultimately going to effect what we do here at PCC, so it might be of interest to you for that reason. But you might also find some sort of emotional resonance in seeing the agency that pressured us to hasten our assessment process now experiencing it's very own pressure from this advisory committee.

If you are a good and compassionate person, you may feel some sympathy for them. Or, if you are another kind of person, you may have another kind of reaction.....

Occupy and Higher Ed: The Cost Accountibility Movement

by Shirlee

The Occupy Wall Street movement has drawn attention to many problems that have been with us for a long time, but without achieving headline status until now. Wealth inequality and its effect on democracy, for example. The distribution of profits in the financial sector, pointing to the question about what is contributed to our collective well-being to justify such profits. And the increasing likelihood that the next generation of adults will not be able to achieve the same level of prosperity and security as their parents....

In connection with that last concern, Occupiers have been drawing attention to the major shift in the funding sources for higher ed over the past three decades -- from grants and scholarships for students, supplementing relatively generous public funding of the institutions themselves to increasing reliance on student debt. The accumulation of student debt is being likened to indentured servitude, even a kind of debt-slavery, by Occupiers. Average debt for students achieving a bachelor's degree is cited as between $23,000 and $25,000.-- and this at a time when unemployment is running high.

So why does higher education cost so much? Where does the money go? These are questions that belong in the Cost Accountability wing of the Accountability Movement. PCC's Learning Assessment Council exists in response to the accountability movement, and the effort to respond to the spiraling costs of Higher Ed has been part of our program from the start. As we here at PCC are busily assessing student learning outcomes, with an eye to making sure our students are learning what we promise to be teaching them, the plan is to locate and share around the instructional practices that are most effective. In this way we will be able to serve more students, more effectively, with less cost per student -- the 3-sided demand on Community Colleges set down by the Obama Administration. The hope is for the quality of our instruction to go up (as measured by achievement of learning outcomes) even as the cost of providing that quality goes down. This is our indirect response to the problem of student debt. Is it enough?

Inside Higher Ed has a piece on the Occupiers' anger about college debt. And in response to student agitation about college loan debt, the Obama Administration has posted a description of its plans. See: https://wwws.whitehouse.gov/

But if you are just plain interested in why Higher Ed costs so much these days, here is the source I recommend:
Delta Cost Project.

Here is a juicy tidbit from page 17:

Among public institutions, spending per student for instruction declined between 2002 and 2005, most dramatically in public community colleges. When state funds increased in 2006, instructional spending increased as well, but not enough to make up for losses in prior years.

Delta Cost Project has charts and graphs comparing costs and funding sources of community colleges to other sectors of Higher Ed. Student debt is increasing partly due to increased costs of education, but the larger factor is what they call "cost shifting" -- shifting from taxpayer funding to individual funding. The increased costs attributable to instructional versus non-instructional institutional costs are also charted. I think it is interesting reading.

We work all day, long and hard, on behalf of our students. It is important to me that we are not just saddling them with debt that will decrease the quality of their future lives -- instead of increasing the quality through all those intangibles associated with being an "educated person." For example, I try to imagine my life without my college experiences, and I cannot do it. College set me on my life's path, and made me into the person I think of as myself. But I graduated with no debt at all. I paid my way by working, with the help of grants and scholarships. That is near impossible these days... Would I be so pleased with my education if it had taken me 30 years to pay it off?

I fear that higher ed looks different with a monthly debt payment that appears to go on forever, and with no good job prospect in sight. I think as educators, we need to know where the money goes -- the money that is at least in part being spent today on our paychecks, but will be paid back by our students long into their futures. And we need to know that it is not just knowledge and skill we are providing our students.... it is years of a repayment plan, too.

Grading Gets Outsourced to India

by Shirlee

The way PCC is approaching the new demand for assessment in Higher Education is only one of several different models that have been adopted across the landscape of colleges and universities in the U.S. These differences became apparent as members of the Learning Assessment Council started to scout around in our first year of existence (2008-9), which was provided to us as a "year of inquiry." Here are short descriptions of some of the ways colleges and universities have decided to go:

1) Many institutions have adopted a single high-stakes standardized test, designed to measure core learning outcomes like communication and critical thinking. These are usually "value added" measurements, intended to show how much development there has been in student competency from the first term of entrance to an exit with a degree. These tests allow comparison of one institution with another, which is part of the call for "accountability" mentioned in last week's blog.

This first approach is based on an assumption that assessment of student learning is a separate kind of activity from instruction. Teachers may be skilled at teaching, according to this thinking, but they are not experts in measuring learning. Measurement expertise is called psychometrics. Experts provide the design and on-going re-design of the major competing high-stakes tests used by colleges and universities in the first model.

(2) In the second model, the idea continues to be that we should leave assessing of learning to the assessment experts, in order to disrupt teachers' lives the least amount possible. In this case, though, assessments are customized to different SACs or departments via the work of a team of psychometricians who are called in to (i) interview faculty about their specific student learning outcomes, and then (ii) design customized assessments to be used by all instructors in that subject area. In this way, for example, experts might come and consult with the PCC history SAC to determine what "critical thinking" means in the area of history, across PCC's history curriculum. Then all the instructors of a given section of history class would be required to administer the test the psychometricians came up with, and the results would be examined to see what they say about the effectiveness of history instruction at PCC. This model gives assessment results that can be used for continual program improvement (the other main purpose of assessment, as mentioned in last weeks blog). The major company that has emerged to do business in this second model is EduMetry.

(3) Many institutions created a new administrative office, and put someone in charge of organizing faculty assessment work. Often, this office oversees the adoption of an expensive assessment software system, and then trains faculty (usually department chairs or supervisors) in how to use it. The software system ensures consistency of reporting, and eases bundling of assessment results for display to the accrediting bodies. For this approach to work, the administrator has to have the power to compel reluctant faculty to both do assessments and then learn how to report results using the system. Faculty are involved to a greater extent in assessing than in either of the first two models, but they tend to be viewed by administrators as reluctant participants likely to drag their feet....

(4) Some colleges and universities have decided that assessing is a critical component of the instructional process, and must be kept as part of the bundle of teaching tasks. The idea here is that faculty are deeply invested in successful student learning, and when they see the connection between assessment and improved learning outcomes, they will embrace assessment as a new and useful tool for doing their important jobs even better. This model then leaves assessment in the hands of faculty, in the form of an assessment committee or council. PCC was set on this path through the recommendation to the college made by the faculty Learning Assessment Council that program/discipline assessment be the responsibility of SACs, and implemented as an ongoing component of Program Review. This last model is the only one that is fully respectful of faculty professionalism and expertise....

The national body for the union that represents PCC's instructors and APs has endorsed this last model, coming up with an interesting slogan that in higher ed we should count what counts. I remain deeply convinced that this last model is the best both for students and for teachers, in the long term. But I am also aware that some faculty at PCC would have picked one of the other models, had they had the choice. And I often call to mind a participant in one of our first assessment classes who voiced a very strong positive response to the second model above, and the company that is most successful in that endeavor, EduMetry.

EduMetry has a varied approach to assessment activities in Higher Ed, and I recently came across another aspect of their business plan in the Chronicle of Higher Education. EduMetry has started outsourcing grading to India through their program called Virtual TA. (See http://chronicle.com/article/Outsourced-Grading-With/64954/) In this part of their business, they devise rubrics for assignments, train and norm a group of assessors on use of the rubric, and then ask their assessors to provide detailed, rich feedback on student papers -- feedback of the sort we all might dream of providing, but are often too busy to actually do. One sociology instructor at a community college, is quoted in the article.

And although Ms. Suarez initially was wary of Virtual-TA—"I thought I was being replaced"—she can now see its advantages, she says. "Students are getting expert advice on how to write better, and I get the chance to really focus on instruction."

It is a new world of assessment in Higher Education. With so many things changing so rapidly, and with many different kinds of responses to the changes being pioneered at different institutions, tuning in to assessment news provides lots of surprises. I used to think that education was a service that couldn't be outsourced. But EduMetry has surprised me. The logic of it is just an extension of the thinking that leads to the first two assessment models I described above -- if teachers are experts at teaching and psychometricians are experts at assessing, and we should each do what we are experts in, then assessing should be peeled off from the work of instructors and handed over to someone else.....

I heard an instructor say the other day that grading was the least satisfying part of his job, and he wished he could teach without having to grade. I wonder if he would really be so happy if EduMetry granted his wish.... Instead of doing more assessing, like we have asked instructors to do at PCC, the day may come when we will do no assessing at all. In the back of my mind, I can hear David Rives (president of Oregon AFT) talk about the de-skilling of the instructor's job...

I say that perhaps we should be careful what we wish for....

Two theories of Assessment

by Shirlee

As assessment has become ever-more-prominent in education and the non-profit world -- a trend that has been building for the past 25 years, according to my reading -- two distinct sets of reasons for assessing have been given. Often they live side-by-side in uneasy alliance. But they are very different, and that difference could make a BIG difference for faculty lives.

Assessment Theory #1 -- ACCOUNTABILITY

The problem assessment is supposed to solve:
Lots of money goes into education and non-profits, and it is hard to tell if it is well-spent or wasted.

The thinking: Lots of money, for example, is given to feed the hungry. Is it being spent well? Are there fewer hungry people than there would have been without the spending? Could it have been spent more effectively? It is hard to say.... Education is in this same boat -- lots of money gets spent, but how can we tell if it is being spent effectively? In this thinking, this aspect of the non-profit/education world contrasts sharply with commercial enterprises. That is, in a market environment, there is a straightforward way to tell if an investment was a good one from a business point of view -- did it lead to the creation and selling of a good or service that enough people wanted to buy so as to make a profit? In a market, it is possible to compare investments over time, using the metric of return on investment, to judge whether or not money was well-spent. But in education, as in most of the not-for-profit world, it is harder to tell what is "working" and nearly impossible to tell if one kind of investment in education -- for example, college scholarships to kids from poorer families -- yields a better or worse return on investment than another strategy -- for example, full-day pre-kindergarten for kids based on income eligibility.

In this context, assessment is intended to provide metrics that enable valid comparison of one possible use of funds with other possible uses. In the world of higher education, this leads to the call for standardized exams of basic skills -- usually writing and critical thinking -- to be given to all students (or a representative sample of all students), across all schools. The scores on these standardized tests would allow easy and clear comparisons of one institution with another. Indeed, there are several exams that are competing to fulfill this dream, such as the CLA (or for community colleges, CCLA) or the ACT CAPP. Lots of colleges and universities have responded to the call for assessment using this theory of assessment by mandating the use of one of these tests.... After the admistration gets the results, they then inform faculty of how well (or badly) they are doing. Poor results lead to lots of administration pressure on under-performing faculty.

Assessment Theory #2 -- CONTINUAL IMPROVEMENT

The problem assessment is supposed to solve:
Our world is in dire need of the skills and competencies characteristic of educated people -- primarily communication, collaboration, and critical thinking skills -- and this mission is so important, we must devise ways to identify and quickly roll-out best practices.

The thinking: teachers tend to be intrinsically motivated, curious innovators. However, they have historically worked in great isolation. One teacher might informally track what works and doesn't work in his/her classroom -- and make changes based on observations -- but this is primarily a matter of independent practice. Education is so important in our changed world, though, we have to take these kinds of individual practices and formalize them. In this way, assessments allow us to formally track the results of innovations, and more quickly and easily isolate components of the most effective education practices, so they can be shared more widely.

In this context, assessments must be in the control of faculty members, as it is their familiarity with their student population, curriculum, content, and current practices that leads to the determination of what to assess. Standardized assessments for a general population of students will not help (for example) history faculty to discern best practices for critical thinking in history. Faculty must control the assessment process, because they have the detailed familiarity to know how to focus an assessment.


At PCC, we have been firmly within the fold of Assessment Theory #2. It is, after all, what our accreditors have asked of us -- evidence that assessment of student learning is being used to improve both teaching and learning. It is also the process that is most respectful of faculty, so it is not suprising that a faculty Council would come up with this sort of a recommendation. Additionally, it is the only approach that could lead to results that would actually be useful to instructor practice.

Notice, however, that Theory #2 leads to ever more customized and distinct assessments, while Theory #1 leads to ever more generalized and standardized assessments. These two theories lead to incompatible pictures of what GOOD ASSESSMENT looks like. The more locally useful a particular assessment is to a given SAC, the less useful it will be to compare one college to another....

I mention all this because one of my heroines in the Assessment World is Trudy W. Banta, editor of Assessment Update: Progress, Trends, and Practices in Higher Education. In the most recent edition (Sept-Oct 2011) she has written a piece warning that "... the promise of assessment for improvement might be diminished by increased focus on assessment for accountability."

She writes:

...[A] significant portion of US colleges and universities may be moving in the direction of providing to the public information based on scores of standardized tests of generic skills that inevitably will be used to compare quality of institutions. It just seems to be human nature to hone in on those easy numbers when we seek a standard for making comparisons."

I offer two thoughts in this context:
(1) We, here at PCC, are lucky to have both an administration AND an accrediting agency operating from Assessment Theory #2. This is the approach that fully respects faculty as THE key players in driving program improvement. Although assessment takes up hours and energy, when done by faculty (and done well) it leads to results that make a difference in student lives -- it leads to continual program improvement and better learning outcomes.

(2) While we here at PCC are getting better and more proficent at assessing our programs and disciplines -- witness the amazing variety and innovation of assessment approaches across our SACs, with ever-better strategies and instruments -- it simply may not be enough to stave off the rush to standardization. At this point, there is still a national push for assessments that fit the picture from Theory #1. Assessing to compare one institution with another is very, very, very different from assessing to be able to do our important job ever better. While both are part of the rise of assessment, they lead to very,very,very different pictures of what good assessment looks like. A one-size-fits-all test looks ridiculous on one model, and the only thing that will work on the other....


Wait. I really meant:
very, very, very, very different.....

I'll keep an eye out and let you know what I see on the standardization horizon. Until then, we will continue with our PCC plan of asking for a splendid locally-controlled profusion of SAC-specific assessment!!