Overview of Research Methods

The emphasis of the chapters in this section of the book are in response to my students’ persistent desire to conclude that their programs caused the program outcomes when neither their research design nor their data would allow them to do so. Even if not in their own work, the assessment plans they produced in assignments were always about wanting to prove that their short-term, voluntary intervention caused an outcome. The goal of these chapters is introduce the concept of research design and what kinds of claim can be made from different research designs.

This part contains four chapters on research methods. The first, Chapter 15 introduces some general fundamental design issues. Chapter 16 covers descriptive designs used when you have data from program participants, and Chapter 17 focuses on designs appropriate when comparison groups are available. Chapter 18 briefly describes qualitative, mixed methods and concluding thoughts.

Bunny Willits, my statistics teacher in graduate school, used to say that statistics don’t lie; statisticians do (or they can). The point is that social science research methods are not inherently good or bad; they can be used for good or ill. As noted throughout the book, traditional social science research methods are based on assumptions that should be understood.  It is incumbent on those of us engaged in the work of teaching about and doing assessment and evaluation to engage in this work thoughtfully, critically, and with equity in mind. Being able to do so begins with understanding the assumptions inherent in assessment tasks and asking how those assumptions influence the outcomes created, the questions asked, the data collected, the conclusions that can be made, and uses to which we put assessment and evaluation results. This is true whether using quantitative or qualitative methods.