Final Evaluation Report - Phase 3

Submitted by

Michaela Hynie
York Institute for Health Research,
Program Evaluation Unit
York University

May 2013

Download complete report (PDF)


Executive Summary

The Canadian Mental Health Association (CMHA) is a Canada-wide charitable organization that promotes mental health, and supports resilience and recovery of people who are experiencing mental illness. Since 2007, the CMHA-Ontario has been running the Minding Our Bodies program, which builds capacity in the community mental health system to promote and evaluate programs promoting healthy living, with the goal of reducing both the rates and the impact of chronic illness. In the spring of 2011, the CMHA Ontario contracted the York Institute for Health Research (YIHR) to conduct an evaluation of the third phase of their Minding Our Bodies program (MOB-III).

The MOB-III program main goals intended to improve the health and well-being of people with mental health illnesses. These goals were to be achieved by increasing knowledge of the relationship between mental health and both nutrition and physical activity; supporting programs in delivering and evaluating a range of physical activity and healthy eating programs across Ontario; increasing the resources available to those who wish to implement and evaluate healthy eating, gardening and/or physical activity programs; increasing life skills for those with mental illnesses, and their improving social inclusion. This report provides a summative evaluation of the MOB-III activities of the CMHA-Ontario, and summarizes the evaluation findings from each of the community organizations that developed MOB-III seeds programs.

The evaluation suggests that MOB-III strategies to disseminate information about the links between mental health and physical health and nutrition were successful through literature reviews and program information made available on the website; presentations, workshops and Knowledge Exchange forums for community organizations and professional and academic audiences; the completion of 20 seeds programs; and the inclusion of the CMHA team in new projects. An exception to the success of the strategies incorporated in the program appears to E-bulletins which, although being widely distributed, were not widely read.

Encouragement of partnerships through the seed programs was the most successful approach to building capacity for program evaluation. Through partnerships, consultation support for program evaluation was provided, and it required no additional resources from the programs (i.e., time) and met their immediate needs at the moment that they needed them.

The relative success of the consulting support also highlights the main input challenges for this program. Time to participate fully, even if they were motivated and interested in doing so, appeared to be the biggest challenge for participants on working groups and in participating programs. This may also speak to why the e-bulletins were read by such a small number of recipients. Successful initiatives will need to take into account the very real resource limitations of small community mental health organization and to provide resources in ways that do not require increased inputs from organizations’ staff members. Similarly, communicating information to the broader community may need to be done in a way that meets community members’ immediate needs or fits in with existing activities.

The overall goal to improve the health and well-being of people with mental health illnesses were mainly illustrated in each of the seed program evaluations. The evaluations suggest that there were changes in skills, health and well-being for participants; programs reported improved self-reported mental health and physical activity levels, at least in the short-term while the programs were underway, and observed as well as self-reported increased social inclusion. Some also reported objective improvements in physical health, such as weight loss or improved diabetes management. Healthy eating programs showed the greatest changes in knowledge and skills; physical activity programs showed the greatest increases in physical health. All types of programs showed increases in social inclusion.

Most programs seem to be motivated to continue with their programming in some form, and the CMHA is continuing to partner on related initiatives, suggesting that the initial changes may be sustained into the future. While short-term outcomes seem to have been achieved, the long-term impact of these programs remains to be seen. It will be interesting to know whether participating in these programs results in sustained change, or sustained participation in ongoing programs, for members of these organizations. It will also be interesting to see the long-term impact on the organizations themselves, and whether participation in this initiative has resulted in permanent changes in the kinds of programs that they offer.

List of Recommendations

Recommendation 1: Ensure programs have sufficient human Recommendation 1: resources to implement programs and participate fully; or seek ways of building in greater resources for these organizations as part of the overall program strategy.

Recommendation 2: Review content of e-bulletins to make them more engaging, and better targeted to recipient audiences.

Recommendation 3: Enhance opportunities for programs to learn about other initiatives directly from other organizations.

Recommendation 4: Explore content of webinars and participation list to determine how better to match content to needs of participants.

Recommendation 5: Make webinars more accessible to a wider range of participants by making them available on-line.

Recommendation 6: Organize more opportunities for partnership building in the knowledge exchange events.

Recommendation 7: Develop additional evaluation tools for monitoring changes in eating behaviour, and physical activity skills and knowledge.

Recommendation 8: Create more flexible schedules for participation in on-line courses.

Recommendation 9: resources or support for evaluation.

Recommendation 10: Invest energy in developing and supporting the community of practice.

 

AttachmentSize
MOB Phase 3 Evaluation Report1.97 MB