What does program assessment look like?

Here are some photos of members of the math SAC working together, using a process offered by the PALs -- Program Assessment for Learning. The PALs are a sub-group of the Learning Assessment Council that can help SACs move from just tabulating assessment results to figuring out what those results mean about the quality of your program, and especially where the good job you are doing now can get even better..... This is what our accrediting agency wants of us -- not just jumping through ever more meaningless hoops, and the filing of ever more silly reports. Instead, SAC members have the power to create an assessment instrument that is designed to answer your own questions about how effectively you are advancing student learning.






Want to schedule a session of the PALs for your SAC? email Paul Wild at pwild@pcc.edu or Sally Earll at sally.earll@pcc.edu.

data-driven educational practices

Michele Marden is a Math Instructor at Sylvania, and a member of the Learning Assessment Council

Study Skills and Self Regulated Learning: Can students self assess? Can we help them do so?

Occasionally a student asks me for advice about how to study math. My responses until very recently were along the lines of “work more problems” or even the less specific “study more.”

Recently I have had a realization: I don’t know how to help my students overcome their struggles because I never had significant struggles with math and I have had precious little training (ie, none) in how one best learns math. This is complicated by the fact that I enjoy math and understand its value while many of my students do not. It is embarrassing that it took nearly 13 years for this realization to solidify!

Last year on a five-hour flight back to Portland, I read a study skills book. It discussed general studying tips and different studying methods for various subjects. Some of the suggestions were obvious, others were not. All went beyond my most common suggestions of “work more problems” or “study more.” None of them were ideas that I shared regularly with students, until now. For my lower level classes, I incorporate “study skills 101 for math.” Sharing study skills has certainly helped some of my individual students, but I always wonder if there is something more I could do…

At a recent math conference, I attended a session about Self Regulated Learning (SRL) given by Lawrence Morales at Seattle Central Community College. SRL goes way beyond my study skills 101. From my brief introduction at this session, I believe that the SRL helps students recognize and handle lack of motivation issues and also gives them tools to evaluate their learning. Note: SRL can be used for all disciplines/programs.

Self Regulated Learning might be a tool for faculty to help students recognize their lack of progress (or lack of motivation) through a framework that would let them self-correct before they are so discouraged that they quit college. Morales described incorporating SRL in his classes as peeling an onion: It has to be an ongoing process where students are trained to self-regulate as opposed to discussing it in one or two class meetings.

SRL is research based and claims to increase student learning. Below is some of the data analysis from a slide Morales shared at the conference session from a CUNY study.

[Data is from slide 16 of the powerpoint from the session. See http://bit.ly/orwasrl for the entire powerpoint given by Morales.]

The CUNY Study: Results

Developmental Students	Self Regulated Learning	Control
Completed Course	73%	67%
Passed final exam	54%	34%
Passed course	50%	33%
Passed COMPASS post-test	47%	27%

Intro Math Students	Self Regulated Learning	Control
Passed final exam	73%	50%
Passed course	68%	49%
Passed COMPASS post-test	64%	39%

Intriguing results!!!
Maybe this will help me give some additional support that will help struggling students succeed.

Suggested readings by Morales:

• How Learning Works by Ambrose
• Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64-70.

If you know about SRL or are interested in chatting about it, please post a comment or contact me.

Kendra and Shirlee invite you to help figure something out!

Kendra Cawley is PCC's dean of instructional support and Shirlee Geiger is current chair of the faculty learning assessment council

Here is a question you might find interesting:
Should all PCC SACs be responsible for assessing for all PCC core outcomes?


Background for the question
Last fall, PCC SACs were asked to create 2 year plans for program assessment. Lower Division Collegiate SACs were asked to focus on the core outcomes, and assess for communication and one other (of their choosing.) Career Technical SACs were asked to pick their "biggest" degree or certificate, map the degree/cert outcomes to the core outcomes and then assess their students for the degree/cert outcomes. 93% of all SACs filed their two-year assessment plans in time to get feedback from the peer review session held in November. 40 faculty from across the district met to talk over the plans they had been assigned to read, in teams that spanned all our common divides -- LDC and CTE, full time and adjunct, big campus and smaller campus.

In that session, as at other points in the process this past year, faculty asked the question:
do all SACs have to assess for all core outcomes?

One reason for answering that question with a resounding "Yes!" is that the core outcomes are what we all have in common - they are what define our shared purpose and our ultimate promise to students, the community, and the tax payers who have supported us so generously by passing our bond proposals.

One reason for answering with a resounding "No!" is that the very point of locating program assessment in the SACs is the firm conviction of members of the Learning Assessment Council that assessment activities must be meaningful -- and the only hope for assessments to be meaningful is that they be created by faculty, based on what we want to know. If a given SAC doesn't see itself as teaching to a particular core outcome, assessing for it would just be silly and a waste of everyone's time.

Or maybe your answer isn’t so resounding, but conditional, or situational, or even uncertain. Tell us why.

You will have a chance to hear more about this issue at the 3rd Annual Assessment Circus, coming to Cascade Campus on May 20th, 9 SM to noon. Anyone who pre-registers will be sent a follow up survey to record their preference. (A later follow-up will go to all faculty, but the results of the "informed" group will be tallied separately.) You can also post your ideas here, by using the comments feature....


We hope to see you there!

Shirlee and Kendra

Assessing for Excellence by Alexander Astin

by Shirlee

An institution's assessment practices are a reflection of its values.

I came across this statement -- offered as "a basic premise of this work" -- in the opening pages of the book Assessment for Excellence, by Alexander Astin. There is lots and lots and lots (and lots) of stuff written about assessment these days, so it takes something catchy to keep me reading. I am a values-driven sort of gal, so the idea that we can tell what someone cares about in education, by looking at how they do assessment -- well, it kept me reading.

Astin says that the word "assessment" covers two very different activities.

The first is basically a kind of measuring. He says most faculty measure things as a form of record keeping -- because they are required to do so as part of their job. We give tests, and record the scores, and are thereby able to calculate grades and turn them in..... This is assessing as measuring.

The second assessing activity requires using the measurements for individual and institutional improvement. In this activity, we ask what to make of what we measured. What does it mean? This is inherently evaluative, and requires a clear-eyed understanding of basic educational purposes and motives.

Many SACs have gathered their items to measure, at this point in the annual assessment process. Some have already done their measuring. If assessing were just measuring, we would record the numbers (as the good record-keepers we are), and move on to something else. But we are now asked to turn to the deeper, more important aspect of assessment -- asking what the measurements mean, as we keep a focused eye on the point of all our work: student learning.

Astin also contrasts three different ways of understanding "excellence in education."

(i) He says some people think the best colleges are the most resource-rich. They have beautiful campuses, hefty endowments, big sports stadiums and incredible labs... and with these resources, they attract the students with the highest scores, and then charge them a hefty price.

(ii) He says the second notion of "excellence" leads to roughly the same listing of "best to worst" -- but based on the idea of "reputation." The schools that attract the most talent (and can thereby turn away the most talent) are the best by a reputational standard.

(iii) But the best way to measure "excellence" in education (according to Astin) is neither via resources nor reputation. He says what matters is "the development of talent." Those institutions that lead to the most actualization of high potential are the best.

Here at PCC, we are asking faculty to develop their skills not just in measuring and record-keeping, but in evaluating what they have measured. What do our scores say about how we are doing by our students? How can we use these results to serve our students even better?

In this, faculty will need to develop more meaningful talents than those of keeping good records. We must, in collaboration, look deeply at our measurements as searchers-after-the-meaning. The meaning that matters is the meaning that helps us excel... Even as we develop new talents (beyond record keeping!) in ourselves, we will be finding ways to better develop the talents of our students.

How we do assessment says who we are and what we care about, says Astin. Like him, I care about people becoming their very best selves... And, following his lead, I like what our assessment process at PCC says about who we are.

Who are we?

We are the ones working hard to develop the talents most needed for our troubled and complex world, so that the life of those who come after us will be better for us having been here... We are working hard to make sure we do this job, and we do it well. We are assessing for excellence. Just like Astin says we should....

Thank you for all you do....

The Completion Agenda

as discovered in New Orleans by
Shirlee Geiger

Last year at the American Association of Community Colleges annual convention, held in Seattle, I felt right at home. For one thing, coffee stations were everywhere, and the caffeine buzz could be heard several blocks away. This year, in New Orleans, it was very hard to find good coffee. Even the famous stuff with the chicory (served with a side of powdered-sugar-sprinkled donuts) was a long walk away, and the place didn't even open until 9. I was astounded by the cultural differences between hot, muggy, and slow Southern life there on the banks of the Mississippi and hyper-tensive hurry and-get-it-done-yesterday Pacific Northwest life we live here on the Willamette...

I was also astounded by the difference in the visibility of Assessment. Last year, in Seattle, there were more sessions on assessment than I could attend. Sylvia and I had to split up to get to them all... This year, there wasn't a single session with "Assessment" in the title. I was feeling a bit forlorn (as well as sleepy, without my daily dose of caffeine), but eventually realized that assessment had not left the scene. It had just receded to the background as an assumption so widespread it didn't need to be mentioned.

Of course, we assess, doesn't everybody?

So what was front and center at the AACC this year? The Completion Agenda. It was everywhere, as in ubiquitous. All over the darn place. There was no way to get away from it.

If you want a cool and clever interactive widget sort of introduction to the problem that the Completion Agenda is supposed to address, please go here:
http://completionagenda.collegeboard.org/

Don't keep reading until you take the test for Oregon.
.
.
.
.
.


Did you take it? (Really, I think you should.... It won't take long.) (That's the kind of assurance needed for hyper-tensive caffeine-fueled Pacific NWers like me.)
.
.
.
.
.

OK, now that you have played with the widget, let's think about what this means for us.

The Obama Administration's Education Department wants a higher rate of our U.S. population to not only attend college, but to finish college -- having earned meaningful degrees.

This means that more of our children need to make it through High School.

But graduating High School and being ready for college are not (as we all know) the same thing. Does the push for HS completion mean more remediation needed at community colleges?

Well, if it does, that is a problem.... There is a move to fund colleges -- not on the traditional measure of how many students are served, described as fte -- but on the basis of degrees awarded. This is a global trend. Here is a place to take a quick tour: completionbasedfunding.pdf. Since remedial classes take up limited financial aid without generating credits toward graduation, there is the risk that as less college-ready HS students are funneled into college, the college completion rate will plummet. If funding is tied to completion, then our funding will plummet....

But there is another problem, as well. One speaker mentioned a study (and I don't have the reference, sorry... not enough coffee) of people who had completed the credits for an associates degree, but never applied to have the degree awarded. In follow up interviews, the people said that an AA or AAS would not do anything to increase their desirability in the workplace... why bother picking up a degree if it doesn't help? So are we being pushed to award ever more degrees, at the same time that our degrees have less and less purpose and use in the workplace?

In our new world of Higher Education, on display at the AACC in New Orleans, it is clear that assessment of learning outcomes is the norm, and the tie between degrees awarded and outcomes assessed is ever tighter. Now comes the sound of the funding shoe dropping... assessments tied, via completion rates, to budgets.

It is the kind of thing to keep a person up at night.... even if you are not wired from drinking too much coffee...

Learner Responsibility in Learning Assessment

By Cara Lee, Mathematics Faculty, Part-Time

Cara teaches at Cascade, and has presented lively sessions on student empowerment in PCC's Teaching Learning Centers

With fierce determination I stood in front of my Math 65 class last term at the SE Center in week 5. I had been feeling increasingly frustrated and drained after each class as the responsibility of the planning, facilitation and assessment was on my shoulders along with the weight of the learning for 35 students.

In my mind I wanted to chew them out for not stepping up to the plate, not doing their homework and not being fully present in class. I knew, however, that a stern lecture or speech would simply yield another “Whatever.”

Instead, I said, “Do you know what I love about teaching college?” Students started to look up from their papers. “You are adults and you are in charge of your own learning. You don’t have to be here, but you are here - by choice.” They gave me their full attention as they sat up straighter, smiled and immediately became more engaged.

In the discussion about learning assessment, I have been trying to figure out where the learners come in. What is their role and what are their responsibilities? Isn’t it their job to learn and to know whether they’ve learned the material or not?

The familiar adage comes to mind, “You can lead a horse to water but you can’t make it drink.” I think this is the mindset of teacher-centered teaching, where the focus is on what the teacher is doing. If the students don’t get it then it’s their fault. They haven’t taken responsibility.

But rather than place all of the responsibility on the students or the teacher, I think the answer lies in the middle. We can form a partnership with our learners on the first day. Agreements are made on the first day whether we do it consciously or not. Most learners are adept at sensing what is actually required of them and whether they will have any control in the process.

In Learner-Centered Teaching: Five Key Changes to Practice, Maryellen Weimer shares the way her colleague took the horse metaphor one step further. “He said it was the teacher’s job to put salt in the oats so that once the horse got to water, it was damn thirsty” (p.103).

Many students do not have experience being responsible for their own learning and don’t have the skills to know what they don’t know. It felt like many of my learners in the math 65 class wanted me to pour the water into their mouths and tell them when to swallow, while they complained about the process. But I was the one who had assumed too much responsibility for their learning and that is what I needed to correct.

In class that day I handed out a mid-term self-assessment form for each student to complete. They each wrote their goals for the class in order to determine how they were doing. They calculated their current grade and listed the number of hours they were studying per week compared to the 10 recommended (hint, hint). They listed other resources they were using. They decided whether or not they were meeting their goals and wrote out a plan for the rest of the term.

I didn’t know what was going to happen but I was very impressed by the shift in the class. The learners did accept the responsibility that I gave back to them. The complainers stopped complaining and started working and asking more questions. The biggest complainer became the hardest worker. With the awareness that they had a choice, they made a different choice.

So in learner-centered teaching, what are we as teachers responsible for? According to Weimer,

”We do have an obligation to show (not tell) students the value and necessity of learning. We have an obligation to make our content relevant, demonstrate its power to answer questions, and otherwise show its inherent intrigue. Once a student interest is piqued, we have the responsibility to lead them to all the learning resources they need. As the student learns, we have the responsibility to monitor the process and offer constructive feedback and assessment” (p. 103).

And what are students responsible for?

“Fundamentally, the responsibility to learn is theirs and theirs alone. We can try to force them into accepting that responsibility along with the obligation to grow and develop as learners, but we do them a much greater service if we create conditions and develop policies and practices that enable them to understand their responsibility and that empower them to accept it” (Weimer, p.104).

I work hard to show interesting and relevant motivations for my content, and I also think that if I am the hardest worker in the class then there is a problem. Some of these shifts are subtle, yet very powerful and only the beginning, I think, of the transformation from teacher-centered to learner-centered teaching. My initial response to the learning assessment movement was, “Oh great, more work for the teachers,” but now I think it’s a different type of work for the teachers and more work for the learners – or more ownership at the least.

I highly recommend Weimer’s book, which gives five key changes to practice. The five changes are the balance of power, the function of content, the role of the teacher, the responsibility for learning, and the purpose and processes of evaluation. She gives many concrete examples of how she centers on her learners in her classes.

Inspired by the Anderson Conference I have made three self-assessment tools for my students which you are welcome to use and modify to fit your style. Self reflection is one of PCC’s core outcomes and I am excited about using these tools to show students how to take charge of their own learning. You can find these tools as well as the latest handouts for my TLC talk, Create the Students of your dreams: 3 ways to empower and motivate students at http://www.pcc.edu/staff/index.cfm/1394,12879,30,html.

Now, would someone please pass the salt...

Weimer, Maryellen. (2002). Learner Centered Teaching: Five Keys Changes to Practice.

San Francisco, CA: Jossey-Bass Inc., Publishers.

Across the Wide Divide

by Shirlee Geiger

Lots of PCC people attended the annual convention put on by the American Association of Community Colleges, held this last week in New Orleans. Although this is a conference primarily for administrators, PCC paid for a number of faculty members to attend. Three members of the Learning Assessment Council were there, including me, with our presence paid for by Chris Chairsell, our Vice President for Academic and Student affairs.

I was surprised -- and happy -- to see that reliance on adjunct faculty all across higher ed was brought up in lots of different sessions, and in lots of different contexts. Generally speaking, colleges and universities are being asked to expand our access to non-traditional students, graduate an ever-higher percentage with meaningful credentials (known as "the completion agenda") and do it all without increasing our costs. And how are we to do this? By staying clear on our learning outcomes, collaborating around meaningful assessments (which help us identify which practices get results and which do not), and then rolling out our evidence-based best educational practices.

This is all requiring a new and unprecedented level and kind of collaboration between educators. The Learning Assessment Council is trying to be of help by creating some new opportunities to engage in respectful and productive peer-to-peer interactions. We will be part of starting lots of new Critical Friends Groups in all our major campuses. And we have another group (the PALs --Program Assessment for Learning) ready to come to the SAC sessions where people look at assessment results, to help keep the conversations directed and productive as SAC members talk about what the assessment results imply about needs for program improvements. (It is always easier to talk abut what a SAC is doing well than what is not going well....)

Members of the council are also in the early stages of discussion on how to help get productive conversations going across the adjunct/job-insecure and full time/job-secure instructor categories. This division makes collaboration tricky in lots and lots of ways. The inequity is clearly unfair -- but the justice issue is not going to be addressed in any satisfactory way any time soon. (The resources are not increasing, any time soon!!) The question is how to keep the unfairness from getting in the way of successful collaboration regarding our common goal -- to serve students, and to serve them well.

I will be talking more about one session that I found especially valuable at a later date. Until then, if you want to read the white paper out of which the session came, I suggest reading:
Engaging Adjunct and Full-time Faculty in Student Success Innovation.

It is a remarkable read.

Shirlee