Metrics: an essential difficulty

A reader of my Sustainability Planning Guidebook recently asked me about measuring the outcomes of the planning process itself. Good question, I thought. Perhaps my answer will help others so I include it here. (Changing the name of the person I emailed to protect identity)

Dear Catherine, I have a moment now to consider your excellent question on measuring the success of the planning process. Let me share some thoughts. It's complex of course and depends on many factors as you know. As we don't know one another, some of this may be very basic and obvious. But I like to start with the fundamentals. 1. "Success" of the planning process depends on how your group defines success beforehand (goals and objectives of the process) and then you compare that with the outputs and outcomes of the process itself. For example, a "process goal" might be to involve a broadly representative group in planning to ensure diversity and broad thinking. You would need to define what you mean by "representative" by mapping stakeholders and prioritizing them. After the planning process, you could look back and assess your success against this process goal by counting the number of stakeholder groups you engaged, what you learned from them and how they influenced the vision. Or you might have a "process goal" that the students themselves will be involved in the planning process as a project-based learning opportunity. You could give these students a simple pre and post test to see how their knowledge, skills and attitudes and values changed. In this case "success" of the planning process is defined as substantive learning outcomes for students. Another example would be an "output goal" which might be to have a written sustainability strategic plan approved and endorsed by certain key leaders or associations or groups. You could assess whether this goal was met. So these are a couple of examples of defining success criteria beforehand that can be measured during and evaluated afterwards. 2. Measurement and metrics can be at least of five different types: -Input metrics: measures inputs into the planning process like # of people involved, # of meetings, # of schools benchmarked, etc -Process metrics: measures "how" you do your planning which is based on your values, planning philosophy and goals and can include improving sense of community, learning about sustainability or strategy or the community, diversity, etc -Output metrics: what the planning process produces, usually specific artifacts like strategic plans, surveys results, benchmarking reports, financial analyses, etc -Outcome metrics: this answers the "so what?" question for everything that has come above. This won't be known for months and years but the measuring and reporting framework and schedule should be set early on. For you, these metrics would include educational outcomes for students as well improved operational performance of buildings and facilities, procurement, etc if that is part of the plan. -Impact metrics: I think of this as "and did the world even notice?" This is a measure of actual impact on, for example, social conditions and environmental problems. It also includes actual financial impact on the top line (increased revenue or intangibles like improved ability to attract funding, talent, etc) and the bottom line (decreased operating costs and risks) 3. Finally, I always suggest a hybrid of conventional metrics and sustainability metrics. Conventional metrics will win support from funders or major stakeholders or gatekeepers. These are usual financial in nature or in an educational setting will be about improved learning outcomes and job placement. Sustainability metrics of course point to a broader set of outcomes related to a prioritized set of social and environmental indicators. For measuring the effectiveness of education/training, see the work of Kirkpatrick and the four levels framework. This is a very complete but not exhaustive answer. Measurement and metrics are what I could call an "essential difficulty". Start small and keep it simple. I hope this helps.