I teach a class in philosophy of science. One of my colleagues here at PCC wrote the text I use, and I like it a lot. The class is aimed at helping students be thoughtful consumers of science journalism by giving a basic overview of scientific method and logic in the context of work done in philosophy and history of science. Part of the focus is on "questionable" science -- claims made in mass media or online about non-standard or fringe claims and practices. One intended outcome of the class is that students, out in the world reading or hearing about such claims, will identify the kinds of questions to keep in mind when making up their minds about the latest (expensive!!) cures and potions, healers or fortune-tellers.
Using some stuff from that class, I am offering you a little pre-test.
There are a group
of practitioners who offer a procedure that they say will improve your
life in X ways. The procedure is very expensive. When asked how we can
tell that it is effective, the practitioners offer to make available
the stories of people who earlier paid for the procedure, and say it
made a huge difference in their lives. The practitioners also say that
they have all had extensive training, and hence are experts in applying
the procedure. They can tell when it is working, and know what they are
doing. They do say, however, that belief in the efficacy of the
procedure is necessary for it to be effective. Some of the requirements
are time-intensive and require discipline on the part of the buyer. If
people can't apply themselves and stick with it, the procedure won't
work.
Here is my pre-test for you:
True or False:
Testimonials from satisfied customers are a reliable source of evidence about a procedure's effectiveness.
True or False:
The collective belief in a group of practitioners that their procedure is effective is good evidence that it is effective.
True or False:
We can trust that, if someone has had training in a procedure, s/he is the best (perhaps only) person who can judge the effectiveness of the procedure.
Here is my pre-test for you:
True or False:
Testimonials from satisfied customers are a reliable source of evidence about a procedure's effectiveness.
True or False:
The collective belief in a group of practitioners that their procedure is effective is good evidence that it is effective.
True or False:
We can trust that, if someone has had training in a procedure, s/he is the best (perhaps only) person who can judge the effectiveness of the procedure.
At the end of my class, I have high hopes that most students will be able to see why the best answer to these questions is "false" and will keep that in mind when they are being "pitched" to by people who stand to profit from their gullibility.
In my work with assessment of student learning outcomes, however, I have been dismayed by how many of my educational colleagues don't see the application of these critical thinking basics to the "accountability movement." When few people had access to Higher Education, the only people who could say whether an education "works" were the people who were supplying the education -- the educators themselves. But in recent days, especially as education has been getting ever more expensive, social scientists have been responding to a request for verification. DOES education deliver the X as promised? Especially in Higher Ed , the X includes the skill of critical thinking.... Is it true that people with Higher Ed credentials are better critical thinkers than those without such credentials?
Some people in Higher Ed have been annoyed that the question has even been asked. (I know some people who do acupuncture who feel the same way -- after all the practice has a long and distinguished history.) But some people have found the question tantalizing, and have started using the techniques of social science to answer it...
As my text says, the first step in investigation is to get a clear understanding of the item being investigated. That has been hard -- everyone is in favor of critical thinking, but it is not clear they are all in favor of the same thing. Still, with lots of people asking what that phrase means, some standard answers have started bubbling up. Once it is defined, the next step is to figure out a way to investigate. Lots of people came up with lots of proposals.... and again, one standard way has bubbled up as, if not perfect, then the best way so far. The next thing to do is create a way to do blind testing, with representative samples and respectable statistical analysis of the results.
Then we go look at the results.
We all know, from the history of science, that this kind of investigation is really good for upsetting orthodoxies and pissing people off in positions of power. I think we are past burning people at the stake for publishing results..... but maybe there are contemporary analogues
Science does move slowly, and no one study is conclusive. Still, the results on Higher Ed are coming in even faster than the results on Acupuncture, now that they are both being scrutinized using the scientific method. Acupuncture is looking pretty good for a limited range of applications. How is Higher Ed doing?
If you are interested in seeing the results of this kind of investigation of Higher Ed, one book is indispensable reading. Academically Adrift.
I know in my class, I tell students that if someone making money from selling X is not interested in looking at the results of investigations, that by itself is a red flag to me.... Do you think you are effective at teaching critical thinking, as a college teacher? Want to look at the evidence?
A group of faculty formed a reading club at PSU to go through the results, and have been reading Academically Adrift together -- then talking about what these results of social science scrutiny mean for them, in the classroom. Want to do that at PCC? Let me know, and we will start reading groups in the TLCs!
But you had best be brave.....
