When Evidence-Based Can be a Constraint: Balancing Evidence, Innovation, and Promising Practices
One of the things we are passionate about at C1C is helping organizations make evidence-based decisions—so passionate, in fact, that it is part of our mission statement. While we are committed to evidence-based practices, we also recognize that evidence-based is not the be-all end-all. This may sound like blasphemy coming from an organization that prides itself on evidence-based solutions, but don’t go writing us off just yet!
While we are committed to evidence-based practices, we also recognize that evidence-based is not the be-all end-all.
First, let me clarify what I mean by evidence-based. In this situation, evidence-based does not simply mean grounded in research (we are all about that research!), but to be defined as an evidence-based program or practice there must be an evaluation or study that proves its effectiveness. There are good reasons certain programs, especially in the non-profit and social services sector, may not be considered evidence-based. However, not being evidence-based does not mean they are ineffective. Below are a few reasons why certain programs or services may not be considered evidence-based.
Innovative Ideas. The definition of innovative is to be new and original. If we’re constantly looking for something that has been proven effective, that likely means we are not looking at anything that is completely new and original. In many cases a truly novel idea hasn’t had time to be effectively studied and evaluated. The criteria needed for something to be labeled as evidence-based generally requires multiple studies to be conducted that often take many months, or even years, to complete. Truly innovative ideas simply haven’t had the necessary time to become evidence-based. Therefore, if we focus too much on evidence-based practices, we may miss out on truly innovative and effective ideas taking place.
Small Sample. Many programs only serve a small number of individuals at a time, which can make it difficult to get an adequate study sample. This is especially true for programs that serve a very specific subsection of the population. For a reliable study, you need a sample size large enough to confidently state that the results are not due to random error. Small studies can be conducted on the smaller cohorts of clients, but it may take years of conducting these smaller studies to finally obtain confidence that the results are consistent and generalizable. If we insist the program must be “evidence-based” for it to be used, certain portions of the population may never receive the services they need.
Lack of Funding. One of the major obstacles that can get in the way of a program being considered evidence-based is a lack of funding to conduct an evaluation. Many non-profit and social services programs have limited budgets, with their funds thinly spread and the majority of it going directly towards client services. A thorough evaluation requires research expertise, multiple points of data collection, and hundreds of hours of work—which ultimately is not cheap to do. There are plenty of programs out there that have been around for many years and have demonstrated promising results but have lacked the funding to conduct an evaluation.
Now, none of this is to say that we should ditch evidence-based practices altogether—we’re still big advocates for evidence-based practices! What it does mean is that we need to find a balance. As with most things in life, if we go to either extreme, we end up losing out on something. If we stop caring about evidence-based practices altogether, we increase the likelihood that we may be using ineffective programs. However, if we focus on only using evidence-based programs, we may lose out on new innovative programs and those designed to best serve a certain sub-group of individuals.
There are still ways to determine whether a program is a promising practice and is likely to be effective. This includes assessing the theory behind the program, whether it incorporates other evidence-based practices, and whether it follows some basic common sense. In general, all programs should have a strong foundation grounded in research and it should be easy to understand how their activities relate to the outcomes they aim to achieve. If the connection between activities and outcomes is not immediately clear and there is no research to support their ideas, there may be cause for concern regarding the effectiveness of that program.
Of course, C1C does recommend conducting program evaluations; this can be used to help understand a program’s effectiveness, strengthen its impact, and increase the support of that program – including potential funding. C1C is committed to helping organizations evaluate their programs in order to increase confidence that they are evidence-based, along with helping to find the balance between evidence-based and innovative solutions. If you want to know more about how C1C can assist your organization in program evaluation or identifying evidence-based and promising solutions, feel free to reach out!