Key Topics

• Definitions of key terms: problem, need, and target population
• Guiding questions for a needs assessment: magnitude and nature of problem
• Guiding questions: what can be learned from the research literature, benchmarking, and best practices;
• Appropriate sources of data to help answer questions
• The role of task forces
• Determining when a problem is a Problem in need of action

Introduction

Creating and implementing effective programs begins with a solid understanding of identified problems and the needs of the people or community touched by or experiencing the problem. One way this is done is through needs assessment. Need is usually generated by some sort of problem or as noted below by a “gap” between what is and what should be. According to Rossi et al. (2004) “Needs assessment is the means by which an evaluator determines whether there is a need for a program, what the need is, and what program services are most appropriate to that need. Such an assessment is critical to the effective design of new programs (p. 102).” This chapter defines important terms and describes the fundamentals of conducting needs assessments.

Colleges and universities continually face problems that in some way keep faculty, staff, students—and the institution—from achieving their goals. The primary task of a needs assessment in the higher education context is to identify and describe the “problem” of concern, to determine what is needed to respond to the problem. The information gained is then used, along with information gathered from the research literature, best practices, and knowledge of what the campus is currently doing, to design effective interventions.

There are two conditions under which one would conduct a needs assessment. The first is a “nothing to something” situation (Owen 2007, p. 169). Administrators may observe a persistent problem on campus for which nothing is currently being done or current interventions are not successful in alleviating the problem. In this scenario, needs assessment accomplishes several important functions. It helps administrators to understand the magnitude and nature of the problem, what is already being done to respond, what needs must be addressed, and the kinds of interventions best suited to addressing the problem.

In the second case, programs may exist to address a problem and yet the problem persists. This may require administrators to go back to the drawing board and reexamine the fundamental, root problem, and needs of the affected population. The nature of problems, clients, and the way they experience problems change over time. An intervention that worked five years ago to address a problem might not work today regardless of how well the program is designed and delivered. In these cases, needs assessment can ensure that the underlying problem and corresponding needs are well understood before a program is designed (or revised) and implemented.

Needs assessment is critical to the development of sound programs or policies. As Rossi et al. (2004) state, “The importance of the resulting diagnostic information cannot be overstated” (p. 103). Needs assessment is also one of the evaluation activities college and university units often shortchange. Absence of systematically collected and analyzed information about a problem poses the risk of the problem being misunderstood, programs being developed based on hunches, personal preferences, and resulting in duplication of existing programs and services. The absence of such carefully collected information about the problem can result in programs that are ill-suited to addressing a particular problem or not address the problem at all. Or, sometimes programs are introduced to solve a problem that does not exist. Implementation of programs that are not based on a careful understanding of a problem can result in a waste of precious resources, including time.

This chapter defines important terms, identifies the types of questions needs assessment is well suited to answering, and discusses the methods for collecting data and for drawing conclusions from those data.

Important Terms

A variety of terms is used when talking about needs assessment that merit some unpacking.

Problem

Needs emanate from a problem or gap: e.g., students aren’t graduating, faculty and staff morale is low, or the institution has a hard time recruiting students from diverse backgrounds. The kinds of social problems that face higher education are rarely, if ever, so clear as to have only one possible definition or response. Social problems such as retention, food insecurity, poverty, indebtedness, sense of belonging or lack thereof, and academic success are given meaning by human beings living in a particular context at a particular point in time, and the same is true of the kinds of problems that colleges and universities face.

These mutually shared meanings result from what philosophers call the social construction of reality (Berger & Luckman, 1967). Individuals interpret the world around them through their own beliefs, their own life experiences, and within the context of community norms and values. As a result, these interpretations can vary widely. There are many current examples of how people understand the same problem from vastly divergent perspectives. For example, some view poverty as the result of income inequality caused by structural inequality—kids who live in poor communities get sub-standard education, live in dangerous places, etc., that in turn contribute to bad schooling and continued low incomes. Others see poverty as a result of lack of individual effort and desire to live off the government. Some view wearing masks to combat Covid-19 as necessary to reduce the spread of the Coronavirus and a responsible thing to do while others see the virus as natural, no more dangerous than the flu, and masks mandates as infringing on personal freedoms. These disparate social constructions of a problem lead to very different interventions as evidenced by widely different state and even county responses to the Coronavirus pandemic.

In yet another example, administrators can easily define what they mean by retention on their campuses. Retention is often defined as continued enrollment in the second semester or second year, for example. Retention rates themselves are pretty straightforward and uncontested as long as all of the parties involved agree upon the time frame used to determine retention. A student either continues to be enrolled or not. Retention becomes a problem, however, when retention rates are low or less than desired for all or some groups. The initiating problem in need of solution in this example is that too many students leave college before graduating.

The interpretation of low retention as a problem and its causes are less clear. Depending on the institution’s mission, the problem could be defined as the students a college attracts are simply not qualified to benefit from the education offered by that college and thus are not retained. This institution might see this as a problem of entrance requirements not being high enough. Conversely, administrators could argue that their admissions standards are in keeping with their mission, and the problem is lack of support for students with low incoming ACT scores or high school grade point averages, and that providing support for students in the best solution. These are very different ways of further defining the problem of low retention and what is needed to address it that lead to very different solutions.

Although administrators may have a general idea about a problem when they convene a task force or begin a needs assessment, it is quite common for a clearer picture and mutually agreed upon definition of the problem to emerge (or should) from the needs assessment itself. (See the discussion about  task forces later in this chapter.) Data collection around a presumed problem helps program administrators and other stakeholders define more carefully just what the problem is. The important point is that it is essential for program planners and evaluators to come to some shared understanding of the problem, how it is experienced, and what it means for action. Needs assessment can facilitate coming to this shared understanding.

The methods used to collect data in needs assessment should be based on thoughtful reflection about methods employed, language used, sampling techniques, data analysis techniques, interpretation of results to ensure that needs assessments do not simply reinforce old stereotypes

Need

One of the keys to assessing needs as a basis for effective program planning is understanding what need is and is not. Need is often equated with “want.” Administrators and program leaders ask students or prospective participants what they want and equate what they want with need. What a targeted group wants may be quite different from what it needs. A need is something without which an individual or a group can’t function or function well (Owen, 2007). A want is something people desire (Owen, 2007). On a personal level you might want a new car, the latest iPhone, or a new bicycle even though what you have now functions perfectly well. None of these necessarily represents a need. On the other hand, if your car with 250,000 miles on it breaks down, and it is the only way of getting to school or work, then a new car is a need. You can’t get to work without one.

The Workgroup for Community Health and Development expands on Owen’s definition and conceptualizes need as a gap:

Need can be defined as the gap between what is and what should be. A need can be felt by an individual, a group, or an entire community. It can be as concrete as the need for food and water or as abstract as improved community cohesiveness. An obvious example might be the need for public transportation in a community where older adults have no means of getting around town. More important to these same adults, however, might be a need to be valued for their knowledge and experience. Examining situations closely helps uncover what is truly needed, and leads toward future improvement. (Workgroup for Community Health and Development Tool Box Chapter 3.)

    A personal example illustrates the discrepancy between want and need. Years ago, my colleagues and I were asked to teach some of our courses on the campus of a nearby community college. The college’s dean of instruction wanted her faculty to be able to earn doctoral degrees by taking classes on the college’s campus. We did a survey of prospective students to see if there was sufficient interest to warrant us doing so. By doing this survey we fell into a common trap: we assessed need by asking potential participants what they wanted.  A substantial number of the college’s faculty members indicated they would attend courses if we offered them onsite. These individuals were apparently expressing a want (a desire) not a need (something without which they would fail to meet their goals or without which they couldn’t do their jobs) as only two students enrolled in the course offered. There were more professors for the course than students! When asked, they might have said they wanted a doctoral degree; but when it came down to paying to enroll, it became clear that faculty at this community college believed they could continue to function well without a doctoral degree (at least without the one we were offering them!)

A final note on definitions. You may also hear the word need used to describe a prescribed solution to a problem, as in students need a recreation center or the community needs a new police station. If either is based on a systematic assessment, then one can say it is a need in the sense described here. Otherwise, such a statement could simply be based on one person’s idea of what is needed.

Needs Assessment

Needs assessment is the process by which evaluators or administrators discover the actual state of a problem—the gap between the real and the ideal, and the presence or absence of critical factors necessary for an institution, program or individuals to function well. Speaking more specifically about needs assessment as related to college students, Upcraft and Schuh (1996) define needs assessment applied to students as “… the process of determining the presence or absence of the factors, conditions, resources, services, and learning opportunities that students need in order to meet their educational goals and objectives within the context of an institution’s mission” (Italics in original, p. 128).

Owen conceptualizes need identification as a multi-step process paraphrased here:

1. Identify the ideal state or what ought to be. For example: every student ought to have sufficient food defined as three balanced meals daily. The ideal state in several examples used so far in this book, generally stated, would be that students should be retained, they should not be hungry, and sexual misconduct should not occur.
2. Determine the actual situation. Some number of students does not have sufficient food according to the above definition of adequate food. To determine the actual state, you might ask: How many students suffer from lack of sufficient food? What are the characteristics of students who have insufficient food? Are there certain times that students experience the need more than others?
3. Determine the gap between the ideal and actual. How big is the gap—either in terms of number of students who are hungry, how much or little food they have or both?
4. Identify the reason for the discrepancies. What are the reasons they do not have sufficient food?
5. Prioritize needs that a program should address and the best way to address them. (p. 172).

    As Upcraft and Schuh (1986) and Owen note, the purpose of a needs assessment is to determine the presence or absence of conditions that some group of people needs to function well. Other assessment and evaluation approaches are better suited to determining how well a specific program is designed and delivered to meet the identified need.

A caution! According to the classic definition, need arises because there is a gap between the ideal and the reality—faculty are not staying, students not graduating at expected rates, etc. As such, it is important when doing a needs assessment to not fall into the trap of simply portraying individuals or a particular group as deficient. The purpose of needs assessment is to understand faculty, staff, student reality with respect to the problem in question and in relation to what the institution or unit  is lacking to ensure success. Thoughtful needs assessment can help units provide the best programs to ensure that all student, faculty, and staff are successful. In fact, one might argue that, if done thoughtfully and correctly, needs assessment is an antidote to presumably well-meaning administrators simply deciding what is needed and or simply blaming individuals for problems.

Finally, colleges and universities may engage in needs assessment-like activities without formally labeling them as such. I will come back to this below when I talk about the role of task forces.

Target Populations

 Needs assessment also involves identifying who would and should benefit from planned interventions. These individuals are sometimes called target populations or audience. It is a waste to provide services for a population that does not need them, e.g., a food pantry for students who have a full meal plan. A program can have direct and indirect targets. In the case of campus food insecurity, presumably establishing a food pantry would directly affect food-insecure students, faculty, or staff who qualify for the pantry services. The sponsoring unit would apply a definition of food insecurity that would be used to determine who would benefit from the food pantry. On the other hand, a program to educate faculty and staff about the problem of campus food insecurity serves faculty and staff as direct targets with the goal of educating them to reach students and others who have insufficient food who indirect beneficiaries of the training program. The ultimate object of the overall efforts is to eliminate food insecurity on campus. In sum, programs are designed to have direct effects on some group of people who are the focus of a program. These programs often have indirect effects on others.

Guiding Questions

As noted earlier, all assessment projects should be guided by questions that focus the assessment activity. These questions serve the same function as research questions in a dissertation or research project. Needs assessment draws on a wide variety of methods and seeks to provide answers to several broad sets of questions described below that can help units in colleges and universities to identify the existence and nature of the problem. Answers to the first two sets of questions are essential to understanding the nature of the problem and thus a basis for reasonable solutions.  A response cannot be developed without a good understanding of what the problem is, how big it is, and who is most affected. The questions listed in the sections below are examples and not meant to be all inclusive. Ultimately, answers to the following sets of questions should provide you with an ability to understand: (1) Who, how many people, and how they experience the problem; (2) the gaps between the real and the ideal; and (3) what is already being done and or might be done to bridge the gap. These questions presume that you have some idea about what the ideal state is.

Determining the Ideal

At some point in the process of conducting a needs assessment, an administrator or task force must identify the ideal state for the problem or concern at hand.  For example, what  is the ideal retention rate?).  If standards for expected performance exist, these may be considered the “ideal.” It might be possible to do establish the ideal as the very first activity in a needs assessment or it might be better done in the process of gaining a better idea of how the problem in question manifests itself on a particular campus.

Identifying the Gap or Problem

These two sets of questions are targeted to identifying the gap—the need. Ultimately, you need to compare the described problem—the answers to the following questions—with the ideal state to help you formulate solutions. NOTE: When using these questions, you need to make them specific to a particular needs assessment.

Specifying the Size of a Problem and Who Experiences It

• What is the initial “problem” giving rise to the needs assessment? What is the agreed upon problem that emerges from needs assessment?
• What is the magnitude of the problem? How many people are affected? How seriously?
• Who has the problem? Who is not being served? What are the characteristics of the people who experience the problem?
• What are the reasons why the problem exists?

Understanding the Nature of a Problem or Need

• What is the nature of the problem? In other words, what is the problem as constituents experience it?
• When do people experience it?
• How do they experience it? How seriously are people affected by the problem.
• Do different groups of people experience the problem differently?

Identifying Potential Interventions

These questions address what is required to reduce the gap.

What is Currently Being Done to Address the Problem (if anything)?

• What services currently exist to address a population or a problem? These could be on or off-campus. programs and services.

Understanding How to Respond to Needs Identified

• How have similar institutions approached this problem?
• What does the research literature say about the problem, its causes, and effective remedies?
• What is lacking or what should programs provide to alleviate the need? This may be addressed by knowing the literature on the problem involved.
• What are best practices or appropriate ways of responding to the need?
• What, if anything, is already being done to address the problem?

   With answers to these questions, it is possible to come to consensus about what the problem is and the basis for responding appropriately.

Methods and Data Sources

Methods for collecting data differ depending on the questions a specific needs assessment seeks to answer. Because several of the questions rely on the same sources of data, I group them into two categories that use similar data collection techniques.

Identifying the Ideal State

There are many ways to identify the ideal state. These can include looking at standards set by organizations, referencing to peer group or normative data, benchmarking to star performers and best practices, or by reviewing relevant literature to see what others have said about the issue. However, defining the ideal state also involves judgement. What might be ideal for Harvard on a particular issue might not be ideal for your institution.

Understanding the Existence, Magnitude, and Nature of the Problem

When administrators identify a problem, they often have preconceived ideas about the problem and to its solutions. Programs have a better chance of succeeding if they are based on thoughtfully and carefully collected data rather than on assumptions or preconceived ideas. One of the first questions that should be asked in a needs assessment is about the size and nature of the condition of concern. Answers to these questions will ultimately help the institution or program decide what the need is and how to respond most effectively. This extends to how people experience the problem.

An institution might know that 15% of the student body (and also staff) experience food insecurity, but this information is only partially helpful in eventually formulating a response that will effectively address the problem. In order to plan an effective intervention, the institution needs to know more about how students experience food insecurity. They need answers to questions such as: How much access to food do students have or not have? How are food-insecure students affected? Why do they not have sufficient food? Are there times the problems is more severe than others?

Answers to questions such as these will help administrators to ascertain what the actual state of affairs that they can compare to the ideal. The information will serve as a base for planning interventions that effectively respond to the problem—to close the identified gap. To reduce duplication of effort, it is important to know what, if anything, is already being done to serve the population in need or to address the problem. To answer these questions, one or more of the following data/methods are commonly used:

Existing Data

Existing data is often a very good source for understanding the size and scope of a problem. In fact, you should always begin by looking at available data. Collecting new data can be a difficult and time-consuming task so one should always ask first what data already exist. Most campuses have an office that regularly collects and analyzes data about its students, faculty, and staff. These offices routinely analyze existing data from the student records system, administer national surveys, and develop and administer local surveys.  Many institutions are now collecting reams of data using student ID cards by having students “swipe-in” when they attend certain events. Each of the data sources listed below provides a specific kind of information that might or might not be relevant depending on the problem you are seeking to understand. For example, local data from the National Survey of Student Engagement (NSSE) might provide useful data on student engagement in various types of activities but would not be helpful in providing insights into student attitudes or campus climate or to an issue such as food insecurity. Following is a list of some existing data sources that could be used in higher education evaluations, including needs assessments:

• Comparative (benchmarking) data from the Association of American University (AAU) for universities that belong to the AAU or from other appropriate peer comparison groups.
• In-house academic information system data housing student records and some faculty related data.
• National associations such as Council on Independent Colleges, AAC&U, National Association of State Universities and Land Grant Colleges, and the American Association of Community Colleges may have useful data.
• The US Department of Education Postsecondary Data (https://nces.ed.gov) and the Integrated Postsecondary Education Data System (IPEDS).
• National Student Clearinghouse
• National survey data
  • NSSE (National Survey of Student Engagement) or CCSSE (Community College Survey of Student Engagement)
  • SERU (Student Experience in the Research University Survey)
  • Cooperative Institutional Research Program (CIRP) Freshman Survey
  • Multi-Institutional Leadership Study
  • COACHE (Collaborative on Academic Careers in Higher Education survey) data from multiple administrations (a survey of faculty)
  • Survey of Earned Doctorates
• Regularly administered, in-house surveys, interviews, and data sources, including regularly conducted surveys of undergraduate, graduate student, and faculty/staff satisfaction.
• Campus climate surveys and environment assessments (See Schuh et. al., 2016.)
• Office records. Most offices record some data on those who use their services. (If they don’t, they should.) For example, a health center keeps records on the number of visitors it has, the main purpose of their visit, and it may even keep records on some demographics about visitors. The student housing department likely has records on the number and nature of disciplinary actions. A provost has records on the number of faculty members being considered for tenure each year and success rates. Colleges and universities collect data on many functions that can be used, such as enrollments, credit hours produced, salaries, student evaluations of teaching, to name a few, that could potentially be used in a needs assessment.
• Social indicator data from sources such as the U.S. Census. Data on some indicators are collected locally and periodically over time. From these data, trends can be identified. For example, Census data and its variants (Current Population Surveys conducted more frequently) can alert colleges and universities to changing demographics of a state or region and can highlight workforce trends. Campus crime data is a good example of social indicator data collected locally by colleges and universities but reported nationally.
• Program documents such as program description, meeting minutes, and any previous evaluation activities. If a needs assessment is being conducted on an established program, existing documents such as meeting minutes may suggest what problem the program was designed to solve.
• Community data, for example on number of transports to the emergency room, arrests, use of social service agencies, alcohol sales, etc.

New Data

You may not be able to find the data you need in existing data sources and may have to collect your own new data. This would likely be the case for the food insecurity example. Following are some methods you might use to do so. Both quantitative and qualitative methods are discussed in more detail in later chapters.

• Interviews or focus groups with stakeholders and reviews of institutional documents provide good sources of data about what program stakeholders believe the problem to be.
• Surveys and/or interviews with those who do or could experience the problem can be very helpful. For example, perhaps there has been some grumbling on campus about low morale among faculty and staff. The last climate survey was done more than 10 years ago so there is no recent data. In this case, to specify the problem further, it is necessary to conduct a survey or a series of focus groups to get at the magnitude of the problem and to learn more about the state of morale. Surveys and interviews with target audiences can also provide insight into what participants believe they need to improve or what the university should do to address a problem.
• Surveys or interviews with key informants may be a good source of information for understanding a problem. Key informants are individuals who are in a position to have information about a population that experiences a particular problem. For example, housing staff would be key informants for questions about the extent of alcohol consumption among first year students. The institution’s affirmative action officer (or its equivalent) may be a key informant if one is trying to understand the number and nature of incidents of discrimination on campus. Individuals who work in the diversity office are likely a good source of information about problems and issues experienced by minoritized students, faculty, and staff members. Supervisors might be good key informants if an institution were doing a needs assessment related to staff.
• Existing literature may be of assistance in developing an understanding of the ideal state related to a problem, its causes, and potential solutions. For example, if retention is the problem at hand, there is abundant existing literature on the general nature of retention, what contributes to students to not be retained, who is not retained or retained at lower rates, etc. In other words, there may be no need to reinvent the wheel.
• Interviews, surveys, or website reviews with peers can be useful resources for how similar institutions experience a problem and respond to it.
• Interviews or surveys of experts may also provide information into causes of and solutions to problems. One resource that college campuses have in abundance is expertise. That expertise can be helpful.

   By comparing what you learn from data collection to the ideal state, one can identify more specifically the needs that must be addressed by a program or intervention.

Learning from Existing Literature, Best Practices, and Benchmarking

Existing literature and current practice are good sources of information to inform needs assessment. Interrogating the literature, benchmarking/comparative data, and best practices helps to identify the existence, magnitude, and nature of a problem on a larger scale, but also can help one move toward identifying effective solutions. Published studies, best practices, and benchmarking are key for determining what services are needed or have been effective in addressing the problem and also for setting target goals. How much service is needed? What has worked elsewhere? Potential participants and program stakeholders can also provide useful information to answer these questions.

 In short, the research and best practice literature is a useful source to answer the following:

1. What does the research literature say about the problem, its scope and nature, causes, and/or solutions?
2. Have best practices for addressing the problem been identified? If so, what are they? Are they used by similar institutions?
3. How does your institution or unit compare to peers on “X” dimension?

Literature Reviews

Literature reviews and research syntheses (both those done by others and those done for a specific need assessment) can inform an understanding of the causes of and potential solutions to a problem. A literature review or research synthesis can help people involved in a needs assessment to understand the problem, the nature and cause of the problem on other campuses and in other settings, and potential solutions. It should go without saying that a solution that works at one institution may need to be adapted appropriately to fit another institution’s context.

A good example of the role literature can play is the case of retention. There is a very robust literature on student retention that investigates who is retained or not, the factors contributing to retention, and the nature and effects of programs to improve retention in different types of institutions. The same is true for student sense of belonging, engagement, and involvement. This literature is becoming more and more useful because it increasingly explores the barriers and facilitators to retention, engagement, and sense of belonging for various groups of students such as minoritized students, and women, and for different types of postsecondary institutions. When using this literature to inform interventions on a campus, it is important to be sure that the conditions on which the literature is based reasonably resemble those on your campus. To use an extreme example, it would not be appropriate to use a study based on elite Ivy League universities to inform interventions on an open-access community college without very careful consideration.

Best Practices

A review of best practices and benchmarks is useful for learning what has worked elsewhere to respond to an identified problem and for establishing criteria and standards against which to compare the outcomes of existing and new programs. Google the term “best practice” and you will find many variations, but the essence of the concept is this: best practices are those that have been shown over time to produce desired results or results that are better than those demonstrated by other practices (Bresciani et al., 2004). As Bresciani et al. (2004) note, best practices are usually determined by comparing program performance to a list of criteria leading to effectiveness. “Best practices” may be practices that meet all of the criteria, or they may meet the criteria at a high level. Either way, “best practices” often become the standard against which programs compare themselves and to which they aspire or on which they model their own responses.

You may also hear the term “evidenced based practice.” The term is widely used in the health field to mean practices that research has shown to be effective through rigorous testing. The term has a scientific feel to it that “best practice” may not and has become popular in the K-12 arena. It will likely gain traction in higher education as well.

Benchmarking

Another term borrowed from the business world that is widely applied in education is benchmarking. According to ASQ (n.d), “Benchmarking is the process of measuring products, services, and processes against those of organizations known to be leaders in one or more aspects of their operations.” Colleges and universities, as well as individual programs, use benchmarking when they identify their aspirational peers or best practices among their peers and compare their own performance on defined indicators to that of the identified comparison group. The goal of this activity is to spur improvement or to show performance relative to that of others. The indicators on which institutions and programs compare themselves to others are typically broad quantitative measures, such as retention and graduation rate, selectivity (e.g., average ACT of entering class), and research productivity.

Benchmarking as a Process

Bresciani, et al., (2004) identify several steps in the benchmarking process that are adapted here.

1. Define the outcome to be benchmarked.
2. Identify the indicators of the outcome that you will compare to the benchmarks. That is, what are the critical dimensions of activity you will compare to the benchmark?
3. Choose the group or standard against which you will benchmark achievement
4. Develop plans and means to achieve desired levels of performance (interventions).
5. Compare achievement on indicators to benchmarks.
6. Integrate benchmarking into practice. (p. 44)

   Benchmarks or standards against which to compare performance often come from normative comparisons with peer institutions or professional association standards. Professional associations, such as the Association of American Colleges and Universities (AAC&U) or the American College Personnel Association – College Student Educators International (ACPA), will often identify best practices that become benchmarks. The Council on Standards in Higher Education identifies functional area or office performance standards (e.g., housing, college health centers, etc.). Performance by highly regarded peers often set the standards (benchmarks) to which others compare themselves. Large research universities often compare themselves to a group of their peers on indicators ranging from faculty members’ salaries to credit hour production.

 Benchmarks are established by a variety of commercial organizations as well. Benchworks by Elentra is one examples of an entity offfering a variety of surveys that allow units, such as student housing, or academic programs such as teacher education, to collect data from students and compare an individual campus’ results to a larger set of data collected from similar institutions on key areas of performance indicators. Member organizations such as the American Association of Universities provides data on a whole host of matters that can be used to benchmark performance. The National Survey of Student Engagement will provide selected peer comparison data on its themes or benchmarks. Likewise, boards of regents or trustees or central administrators may specify the attributes or indicators—the benchmarks of performance—on which they collect information to compare units under their charge. I come back to Suskie’s (2015) reminder that numbers only mean something in evaluation when they are compared to other numbers. These “other numbers” often come from comparisons to peer institutions.

Uses of Benchmarks and Best Practices

Tools such as best practices and benchmarking are useful in needs assessment in several ways. Benchmarking and best practices may help programs identify gaps or needs where they fall short of peers. Best practices can help identify how others have approached the same or a similar problem and may enable those conducting the needs assessment to identify interventions that have been particularly successful elsewhere that could be adopted and adapted. Best practices and benchmarking can also provide standards against which a program’s performance can be compared.

A discussion of literature review, best practices, and benchmarking can also help to develop a program logic model (next chapter) by helping administrators understand what kinds of interventions are effective in addressing a particular problem and how they do so.  Additionally, both benchmarking and best practices can be useful for setting performance standards for operation/implementation and outcomes assessment. Although needs assessment is most often done before implementing a new intervention, it may also be useful if a program’s target audience or the nature of the social problem has changed substantially. The nature of problems changes over time and what worked ten years ago might not work today.

Methods of Data Collection: Summary

To summarize, needs assessment addresses a variety of questions and means of data collection. The following table summarizes both:

The Task Force

Needs assessments in higher education are often carried out by task forces or committees. It is often difficult for a single person to carry out the work of a needs assessment. The more universal the concern, the more likely an institution is to convene a representative task force. As Bolman and Deal (2003) note, “Task forces assemble when new problems or opportunities require collaboration of a number of specialties or functions” (pp. 52-53). Rarely does one unit have the capacity or the ability to act alone to define or solve a problem such as low faculty and staff morale, retention, campus food insecurity, or sexual assault on campus. Problems of this nature are usually campus-wide and many different units are involved in or have a vested interest (are stakeholders) in the problem and solutions. The larger and more complex the problem, the more a task force is likely to be created. In a scenario focused on advising, the vice provost might create a task force to take the lead on understanding and defining the problem and recommending an intervention. The task force would likely consist of individuals from a variety of units who have some insight into the problem.

Task forces usually study a problem, collect and review relevant data, and make recommendations for action. In so doing, they are conducting a needs assessment as part of their work regardless of whether they formally call what they do a needs assessment or not. Even if a problem is unit specific, such as a problem involving excessive noise in the residence halls, it is likely that the housing department would create a committee or task force to identify the problem and tackle the task of designing a solution. Presumably if stakeholders are involved in the solution, the solution will likely to be more effective because there will be more buy-in.

There are at least three potential challenges with task forces. One is that of membership. In order for results to be deemed legitimate and helpful, membership should be appropriately representative of stakeholders who have needed expertise. Task forces may also take considerable time to do their work and issue a report. Needs assessments take time and sometimes and some problems require quick responses. Developing expedient ways to collect data to identify problems and needs is recommended. Routine monitoring of key indicators is one such strategy. A third problem with task forces is that they are typically involved on the front end of problem diagnosis and planning for a solution but then are disbanded. Many of their members may not be involved in further roles with respect to the program.

Drawing Conclusions

The information obtained from a needs assessment is used to help formulate a good understanding of a problem (the gap between the ideal and actual) and to identify solutions to the identified problem. According to the logic of program evaluation discussed in Chapter 4, individuals conducting a needs assessment ideally would have determined ahead of time which data to collect about which aspects of the problem and the size of the gap necessary to constitute a problem, such as the number or percentage of students, faculty members, or staff experiencing some sort of problem (e.g., discrimination or food insecurity).

Getting from the data about the problem to deciding whether action is necessary is a bit more complex. There are two approaches to doing this: The first is to set some sort of threshold upfront for the size of the gap necessary to prompt action. Data collected from the needs assessment is compared to the threshold. If data suggest the problem is smaller than the threshold, you might decide you don’t need to act or can plan a smaller intervention.

Ideally, performance standards would have been set for each aspect of the problem, for example, how many people have to experience the problem in order for it to be considered a problem in need of a solution. Often national normative data are used to help set standards for performance (or lack thereof). For example, data from the National Survey of Student Engagement can provide comparative data on how engaged students are on one campus compared to norms for its type of college. Likewise, universities that are members of the Association of American Universities (AAU) use data from other AAU institutions to help them set performance expectations or at the very least to serve as a point of comparison. The AAU has created a data exchange expressly for this purpose. Most types of colleges and universities have their own set of peers to which they compare performance on a variety of measures. These comparisons can help administrators make a determination whether something is a problem, how big, and perhaps offer targets for improvement.

The second strategy is more inductive. You don’t begin with a pre-determined standard for the specific definition of the problem. The data collected helps a task force to understand the specific dimensions of the problem and recommended solutions. Using the example of food insecurity on campus, you might have looked at some data on food insecurity among college students. Let’s assume the nationwide figure is 25% of college students face some form of food insecurity (It’s higher now.) With these national statistics in mind, you analyzed data from your recent needs assessment and decided, based on the analysis, that the number of students who suffer from food insecurity on your campus (let’s say 10%) was lower than the national average. Moreover, students who suffer from food insecurity have two meals a day but not three, which was set as the ideal. Following the first approach, the task force might determine that the college does not need to respond because the problem is smaller than it is nationally. That said, you might not want to ignore the problem and instead facilitate a partnership with a local food bank(s), identified in the process of data gathering.

To be clear, I am not suggesting that administrators be driven by cold-hearted numbers alone even if you do have limited budget and staff resources. Just because the number of food-insecure students did not meet your rather strict definition of a problem, you should not simply ignore the problem or assume that students aren’t food insecure. In fact, the point is the opposite. You should collect data to inform your decision making, recognizing that data alone cannot substitute for informed judgment.

In other situations, you might decide that a lot of students are experiencing a very small problem that is of insufficient consequence to warrant an intervention. On the other hand, a campus sexual assault task force might have identified two reported incidents of rape on campus in the last three years (far below the national average) and yet recommended that intervention is necessary because of the nature and consequences of the problem. Or, in the campus food insecurity example, perhaps only 10% of the students have insufficient food, but they are severely affected by lack of food to the point of having to drop out, making a response necessary.

As evident from this discussion, the determination of when a problem becomes a Problem, with a capital P, meriting institutional response, depends on many factors that cannot always be predetermined and set in hard and fast terms. That is, you cannot, and should not, always rely on numbers alone to make a decision. Making such a decision requires careful, equity minded, judgment often involving a set of comparable data from selected peer institutions as well as familiarity with the context. For example, you could use the retention rates at peer institutions to help your institution establish an ideal retention rate or to make a reasoned judgment about whether your college’s retention rates constitute a Problem. If your retention rate falls below that rate, you have a Problem in need of intervention.

On the other hand, perhaps your retention rate is in line with that of your peers, but the administration wants to do better. In this scenario retention rates are also a Problem but may require a different response. Likewise, you could draw on national data about prevalence of sexual assault on campus to establish the number of cases that would push you to take more significant or different action than already being taken. However, given the seriousness of the potential harm, you might decide to develop a programmatic response even if the campus rate is below the national average. Thus, labeling a phenomenon a problem and identifying the specific need is often a judgment call involving various considerations such as number affected, seriousness of the issue, and the consequences of not acting.

Summary

In sum, needs assessment typically occurs before a program or intervention is developed. It helps program administrators and stakeholders to understand the nature and extent of a problem. Careful assessment of needs provides an understanding of the problem and forms the basis of an intervention that has a potential to effectively address the problem and prevent wasting of resources. A second part of needs assessment involves examining the extant research and best practice literature that can help an institution to further understand the problem but also illuminate how others have responded to the problem and what types of interventions are most effective. Thorough needs assessment is an antidote to the “ought to” and “should” tendencies of well-meaning administrators whose orientation is to implement solutions to solve problems before adequately understanding the problem to be solved. This leads to better programs with a greater chance of success at filling the “gap” between what is and should be. Moreover, times change and problems evolve, as do appropriate solutions. A needs assessment conducted in 2014 might not provide an adequate basis for understanding a problem or those who experience it in 2024. Needs assessment thinking, if not formal process, should become baked into administrative action.