Evaluation is a valuable tool used by BC Mental Health and Substance Use Services (BCMHSUS) to strengthen the quality of our programs and improve outcomes for our clients.
Evaluation informs our commitment to trauma-informed, culturally safe, recovery-oriented, and patient and family-centred programs and services.
Evaluation needs to be differentiated from research, quality improvement, and performance measurement. Despite some similarities, their goals are distinct from each other.
- Evaluation is the systematic assessment of the design, implementation, or results of an initiative for the purposes of learning or decision-making. The goal of evaluation is to examine a program’s processes and outcomes and make evidence-based recommendations.
- Research is a systematic approach of collecting, documenting, and analyzing information to understand a problem or gain knowledge about a specific subject. The goal of research is to generate new knowledge that can be generalized to a wider population.
- Quality improvement is an ongoing collection, analysis, and use of information to manage performance, increase positive impacts, and minimize adverse impacts. The goal of quality improvement is to improve internal processes, practices, costs or productivity for a specific program/intervention.
- Performance measurement is the systematic and periodic measurement of performance over time against key performance indicators (KPIs). The goal of performance measurement is to offer early and ongoing information about a program’s progress towards meeting key indicators and standards.
BCMHSUS evaluates its programs and services to encourage continuous learning and evidence-based decision-making, identify areas of improvement, ensure accountability, monitor progress, and determine whether the program is producing the desired results. Our work in evaluation reflects the actions outlined in the BCMHSUS Directional Plan
, specifically the commitment to advance a culture of learning, innovating, and sharing knowledge. Below are some examples of evaluation:
- The UNITE evaluation: focused on assessing if, and how, stigma is reduced as a result of a mini-documentary series detailing the stories of people living with mental health and addiction challenges; dialogues sessions with staff, clients, and families to engage in meaningful discussions around stigma; and other activities aimed at creating change and promoting understanding.
- Evaluation of the Research Challenge initiative: aimed at understanding how engagement in research activities can increase knowledge of research among staff, connect evidence and practice, strengthen connection between the research institute and the clinical staff, and promote patient-oriented research.
- The Forensic Psychiatric Hospital Action Plan evaluation: intended to identify measurable impacts related to the FPH action plan’s goals, and to assess the progress towards the intended outcomes.
Program evaluation can inform decision-making, policy development, and initiatives and systems. Program evaluation can help ensure that resources, such as human effort, time, and activities are allocated in the most efficient and effective way and that the initiative is working as expected. Evaluation results also provide the evidence needed to secure funding and ensure program sustainability.
Staff are key participants and beneficiaries in program evaluation as they have the closest proximity to the initiative and can apply the evaluation findings in practice.
Clients are the ultimate beneficiaries of the evaluation results. Evaluation recommendations can inform the quality of care and contribute to the improvement of client experience.
BCMHSUS follows the best practices in evaluation that consists of the following steps: identify stakeholders, describe the initiative, develop the evaluation plan, gather credible evidence, justify conclusions, and ensure use and share lessons.
This framework demonstrates the six steps that can be a starting point for tailoring an evaluation for a specific project or intervention. Each step provides a foundation for subsequent steps that need to engage relevant stakeholders.
The BCMHSUS Patient Engagement Framework
outlines the key benefits of engaging patients and families in different activities, including program evaluation. These benefits include, among other things, improvement of health outcomes and quality and patient safety. Engagement of different stakeholders in evaluation increases the utilization of evaluation results, creates an interactive learning environment, and improves the quality of evaluation. People with lived and living experience and their families should be involved at every stage of evaluation, from defining a scope and evaluation questions to analysing the data and developing recommendations and future steps.
The dissemination and implementation of knowledge gathered from an evaluation is critical to inform future decisions, provide actionable steps for improvement, and build capacity. At the same time, KTE activities and their effectiveness need to be evaluated. Common knowledge translation mechanisms include reports, infographics, data dashboards, and publications. Creative approaches that value a power of story-telling are also part of KTE. When considering implementation of recommendations, KTE professionals work with stakeholders to enable the use of evidence to create change. BCMHSUS has developed a KTE Framework to guide these activities.