by Dominique Turnbow
We’re at the end of this series about using the Kirkpatrick Model. In previous weeks, I’ve written about the value of Level 1 evaluation, Level 2 assessment and provided an example of how I incorporate into my workshops here and here. This week, I want to turn our focus to the last two levels that examine behavior and results.
As with previous posts, let’s look at the questions that drive the kind of information that you should strive to answer if you are assessing for these levels. Level 3 is where we focus on learner behavior by asking:
To what degree do students apply what they learned in a workshop to their research assignments?
Level 2 can tell us what students learned, but Level 3 tells us if students are actually using what they learned. Just because students can do it (Level 2), doesn’t mean they will do it (Level 3). Some examples include:
- Are students using the databases you pointed to in the workshop or are they still doing a Google search?
- If they are using databases, are they searching them using the strategies you demonstrated?
- Are they actually using library resources to access materials or are they settling for whatever they can find for free online?
As you can guess, it is difficult to do Level 3 assessment. I would argue that it is not possible to do in a one shot. At best, you would need to be following the students throughout a course to be able to see if they have changed their research behaviors as a result of library instruction. In this situation, you have many options for assessment, including course projects, papers, etc.
Finally, Level 4 assessment is concerned with results. In our environment, we can ask:
To what degree are learners information literate as a result of library instruction?
It is important to note that Level 4 assessment is the big picture. It is not something you could do after one workshop, or even one course. It would be more beneficial to assess at this level at the end of a student’s time at the college or university. It is about return on investment. In libraries, Level 4 looks at the degree to which a student is considered information literate as defined by your institution or the ACRL Information Competency Standards (or Framework for Information Literacy in Higher Education). While it may not be feasible for you to do Level 4 assessment on your own, it is definitely something that you could do at an institutional level. There are several tests developed for institutional assessment of information literacy competencies. While each have a different focus, they all aim to provide data for Level 4 assessment.
The Kirkpatrick Model offers a framework that we can use to evaluate and assess our instruction. While the goal may be to answer the questions for all four levels, there can be freedom (and relief) in knowing that you may not be able to achieve assessment at all levels in your one-shot workshop. It is important for us recognize and discuss (with each other and our administrators) what we can and cannot accurately evaluate and assess. Doing so may open the door to new opportunities for discourse about what our individual and institutional needs are. It is necessary to understand the questions you need to answer in Level 4 in order to design solid evaluation and assessment in Levels 1, 2 and 3.
What do you think about Levels 3 and 4? Is the Kirkpatrick Model useful to you? I look forward to continuing this discussion online via the comments or during our Snack N’ Learn scheduled for next week.