Data IDs Best Practices

by Shirlee

Sometimes I try to describe the accountability movement in higher education in words that can fit into the proverbial nutshell. That's when I rely on an analogy with the medical profession. Here is how it goes:

• The practice of medicine has traditionally been considered a profession where the practitioners (the doctors) are considered to be "experts" who we are all asked to trust.
• As a result, there have been few ways for someone "shopping" for a doctor to meaningfully compare one M.D. with another.
• Even so, it is known that some doctors do, in fact, get better results -- and often at lower costs -- than other doctors, treating the same conditions in similar patients.
• The cost of medical care has skyrocketed of late, and the mechanism we have created for payment (work-based insurance) is leading to huge social disparities, with a clear consensus that something has to be done, even as there is no consensus over what that is.
• Rumblings have been going on for a while now that one way to contain costs and increase access is to assess patient outcomes, in order to identify BEST PRACTICES and then make that information available to patients and taxpayers.

In the above story, we can change all mention of doctors to professors, and patients to students, and everything works the same....Really.

• Teaching in a college or university has traditionally been considered a profession where the practitioners (the professors) are considered to be "experts" who outsiders have been asked to trust.
• As a result, there have been few ways for someone "shopping" for a college or teacher to meaningfully compare one option with another.
• Even so, it is known that some colleges and teachers do, in fact, get better results -- and often at lower costs -- than others, even when the student populations are very similar.
• The cost of higher education has skyrocketed of late, and the mechanism we have created for paying (student debt) is leading to huge social disparities, with a clear consensus that something has to be done, even as there is no consensus over what that is.
• Rumblings have been going on for a while now that one way to contain costs and increase access is to assess student outcomes, in order to identify BEST PRACTICES and then make that information available to students, their families, and taxpayers.

Just like practitioners in the medical field, those of us in Higher Ed are being asked to:

• expand access to our services
• get ever-better outcomes for those who enter our doors
• and do this with less money per student.

There are, however, points of dis-analogy between the two fields.

• There are usually fairly clear indicators of success or failure (like mortality rates)with medical procedures --but success is harder in Higher Ed. If someone takes some community college classes, doesn't get a degree, but does get a promotion at work, is that success? or is it failure?
• In the medical field there are some service-payers that are so large, and who have been keeping records for so long, that there is LOTS of data to be mined. The biggest and best of these data piles comes from Medicare and Medicaid -- but for higher ed, there is no comparable keeper-of-records who could furnish us with data to study. Instead, we are in the early stages, via assessment of learning outcomes, of gathering that data.

Now I mention all this because I read today that the HUGE pile of data on patient outcomes is about to be released, in a format that will make it especially search able. Here is the link, plus a short excerpt:
http://my.earthlink.net/article/hea?guid=20111205/3c47d411-d964-4fce-933a-d1d1111584f2

"The government announced Monday that Medicare will finally allow its extensive claims database to be used by employers, insurance companies and consumer groups to produce report cards on local doctors — and improve current ratings of hospitals.

"By analyzing masses of billing records, experts can glean such critical information as how often a doctor has performed a particular procedure and get a general sense of problems such as preventable complications.

"Doctors will be individually identifiable through the Medicare files, but personal data on their patients will remain confidential. Compiled in an easily understood format and released to the public, medical report cards could become a powerful tool for promoting quality care.

"There is tremendous variation in how well doctors do, and most of us as patients don't know that. We make our choices blind," said David Lansky, president of the Pacific Business Group on Health. "This is the beginning of a process to give us the information to make informed decisions." His nonprofit represents 50 large employers that provide coverage for more than 3 million people."

Notice that the ratings are happening on two levels -- the hospitals (analogous to the colleges) and the doctors (analogous to the instructors.) Many colleges have already taken steps to help create a data set that can be used to compare one institution to another, by using one of the standardized tests (usually of critical thinking and communication) that have been created to allow just such comparisons. Instead of that route, we here at PCC have asked SACs to create or adopt assessment instruments that can deliver info they need to continually improve instruction. This gives us locally useful information, but no way to compare ourselves, as a college, to others. But so far, neither approach (standardized test, customized SAC assessment) will provide a way to meaningfully compare one instructor to another, the way the Medicare info will allow comparisons of one doctor to another.... Still, I say, any data that is aggregated can be disaggregated. And I think it is wise to attend to trends in the medical world, as hints of what will be coming our way.

Some of all of this makes me joyful. The faster we can figure out -- and share around -- what works, the more our students will learn. According to an article Linda Gerber sent my way, there is now more student debt than credit card debt in the US of A. This is a staggering realization. Go read this and weep: http://www.usatoday.com/money/perfi/college/2010-09-10-student-loan-debt_N.htm
But some of all of this makes me wonder how many of the traditional ways of higher ed will be changed beyond recognition in this process. ...

Evidence-based educational practices are a new trend, just like evidence-based medical practices. When my oncologist, 4 years back, laid before me the success rates of various treatment options for my kind of cancer, and helped me poke through the list to decide what to do, I was very grateful for this trend. (Since this pre-dates PCC's insurance for part-time faculty, my insurance wasn't that great -- since it was an individual policy, I didn't get the advantage of group rates -- and cost was one of the factors I considered.) Will the day come when there is an analogous approach to selecting college or college teachers? -- a high school college adviser lays out the same kind of data on rates of learning for college writing or critical thinking, and compares what is available to the student's aspirations and budget?

And should such a day come, how will PCC look as an educational choice?

These are among the interesting questions of our times....