Preface
For the last 15 years or so of my faculty career, I taught a master’s and a doctoral course in program assessment and evaluation to prepare students who are current or future administrators to be informed consumers and doers of assessment. During the early years, I tried various approaches to teaching the course. For the first 5 years, the course was basically an old-fashioned lecture course with a focus on how to conduct assessments and evaluations using different evaluation models. To apply course concepts, I had students work in groups to do actual evaluation projects for offices on campus. Although often fun and interesting, this approach was only moderately successful in helping students learn important assessment concepts. It was hard to round up enough projects on a yearly basis that were relatively equal in scope and difficulty and doable in one semester. To compound the challenges, students had very basic research skills, limiting what they could do well on their own.
One of the biggest challenges was finding a text that had a broader focus than just student learning outcomes assessment and yet was not so technical that students were completely overwhelmed. I tried many different evaluation books and reviewed many more. They were often expensive and written for a specific audience; some were too research method/statistics heavy. Even when the texts seemed straightforward to me, students struggled with the jargon. They got hung up on the certain things, such as “which model should I use?” Almost none of the general evaluation texts used higher education examples.
Our comprehensive exams indicated that master’s students weren’t developing an understanding of, and ability to apply, assessment and evaluation concepts at the desired level. A change was necessary. At some point in this process, I decided that I would try writing my own text. In doing so, I also engaged in a backwards design process to focus on what was most important in an introductory assessment and evaluation course. The resulting book is an adaptation of some of the classic evaluation texts such as Evaluation by Peter Rossi, Mark Lipsey, and Howard Freeman (2004) absent much of the technical language. It is broader in scope than most texts devoted to student learning outcomes assessment in academic and in student affairs programs covering some topics, such as needs assessment and logic models, in more depth. I also try to use examples from a broad range of programs beyond the academic and student affairs.
Perhaps most importantly, given that the majority of students taking the course have limited research skills, I was forced to think about what knowledge and skills are most important for students to take away from the course. The result of that exercise was to refocus the course goals and book on understanding the kinds of questions one can and should ask about the problems they encounter and programs they develop in response. Then, students should have some idea about the types of data they could collect to answer those questions. Rather than focus on statistics or qualitative methods per se (and duplicate the efforts of hundreds of others who have done so), I focus on the logic of research of research designs and what they will allow one to conclude about a program’s effectiveness.
The results of using this book, along with an approach to teaching called Team Based Learning for the master’s version of the course in which students worked in assigned teams throughout the semester to engage in evaluation tasks using material from the book, produced very positive results. Assessment and evaluation went from being a weaker component of the comprehensive exams to being one of the stronger. See Michaelsen, Knight & Fink, 2023 for more on Team Based Learning.
Several groups of people have been important to the development of this text. My colleagues, Dr. Marlesa Roney and Lisa Wolf-Wendel read and commented on earlier editions. Much of the text on quantitative data collection and analysis is loosely based on a yearly lecture that Dr. Wolf-Wendel has provided for the class. My new colleague, Dr. Dallas Doane encouraged me to make the book an open educational resource, something I have wanted to do for years but just never got around to doing.
The current edition is informed by dozens of students, both master’s and doctoral. One of the advantages of writing one’s own, self-published text is that I can change, edit, modify, add, or subtract based on how students interact with and understand various concepts in the book. The book has gotten both better and longer as a result of students who have used it and for that I thank them. Finally, the book draws heavily on Rossi, Lipsey, and Freeman’s (2004) classic Program Evaluation: A Systematic Approach. Their book has had a significant impact on the way I have come to think about assessment and evaluation. Any errors of interpretation are mine as are typos or other error of grammar, syntax, or repetition!
