Blair Dimock is the VP of Partnerships and Knowledge at Ontario Trillium Foundation.
Ben McNamee is the Director of Measurement and Business Intelligence.
Tell us about you. What motivates you? What challenges have you been focused on solving?
Blair: I joined Trillium about 12.5 years ago. We've been on a journey down the continuum of moving from measuring only simple things like activities and outputs, to trying to become an impact grantor with the ability to measure and report on the impact and results in different ways.
Ben: I joined OTF just over three years ago now, as OTF was just beginning that the new strategy of measuring outcomes and measuring impact, and sort of moving away from that activities and output focus. So my background is as an applied economist and so that skill set fit for the shift.
I view this work really as almost an ethical imperative to look at the actual outcomes of all the dollars invested in the social sector - The hundreds of millions, billions of dollars invested - And not much knowledge as to what is actually changing as a result of these investments. And so the state of impact valuation to be able to answer those questions and not simply answering those questions but then doing the next step of learning from that and acting on that as well.
"I view this work really as almost an ethical imperative"
Tell us about your experience with impact reporting? What role do you see OTF playing in the impact reporting landscape? How will OTF’s role work with others in the space?
Blair: In 2011, we spent our billionth dollar as a grant-making foundation. I can remember that day when I saw that number come across my desk. I walked into my CEO’s office and said “Hey do you realize we've just invested our billionth dollar? What a great story this could be for the Province of Ontario and the Foundation!” And the look I got back from my CEO at the time was like a deer caught in the headlights because we had no way of creating a narrative around what difference that billion dollars had made in the Province of Ontario. We could cite in great detail where the money has been spent for every single grant that had been made. But we had no way of really talking about what difference that billion dollars had made in the lives of people living in the Province. So we decided to pivot, to reinvent our strategy with a very clear view in mind. We wanted to be able to tell a very different story after we spent the next billion dollars which would've been another 10 years out. And that was actually a challenge that the next CEO put to us and said “OK figure it out. How are we going to demonstrate and tell the story of what difference a billion dollars makes over the next 10 years in communities in Ontario?” That became the focal point for developing a whole new approach both to what we look for when we're asking applicants to fill out an application form to what we're looking for in terms of how the results that are to be reported and defined, the kind of impact that we’re looking at. We’re being much more intentional about the ways in which we are using data and storytelling differently to actually shift the narrative.
"In 2011, we spent our billionth dollar as a grant-making foundation. I can remember that day... I walked into my CEO’s office and said “Hey do you realize we've just invested our billionth dollar? What a great story this could be for the Province of Ontario and the Foundation!” And the look I got back from my CEO at the time was like a deer caught in the headlights."
Ben: My introduction to impact, and measuring impact is through the Social Return on Investment framework and lens. And somewhat through my academic work in applied economics. The level of rigor required of academic work and social return on investment framework is so high, and really is such a barrier for so many people to access this kind of work. But our context at OTF of being a grantor for the whole province from north to south, east to west, small world communities downtown Toronto etc. requires that the approach we take is something I can fit and make work for all of our beneficiaries.
OTF is in this position of trying to straddle the trade-off between really good rigor and making sure we have confidence in the impact measurement and the impact of our stakeholders and grantees. But, also the recognition of meeting our stakeholders and grantees where they’re at, and what works for them and their context and their capacities. And so, I think there's a situation where the field of impact reporting has gone so far beyond and become really sophisticated in some areas. And in some ways, the role we can play in this field is to help everyone catch up and be that bridge between that the forefront of the field and where most of the sector is.
In some ways what you’re talking about is similar to string theory in physics. Tying together the very small objects with large events like black holes. Being able to bridge the small with the large into one cohesive picture is a challenge. It also sounds like Ontario Trillium Foundation is uniquely challenged, in Canada at least, by scale. What do you think success at scale looks like? In some cases, a grantee may not even know how to define an outcome.
Blair: There are two challenges around the scale. The first is it’s not just about the scale, but it's about the breadth of the scale. So high volume is different than simply a lot of money. A high number of grants, very diverse array in the portfolio. There's a huge challenge. How do you measure and report on impact in a meaningful way for grants that might be $10,000 to do a small project in one year? Versus a multi-year and larger dollar investment?
The solution has been to make sure that this is not a one-size-fits-all approach. We have to be very tailored to the extent we can. I think the other challenge for us is striking the right balance between standardization and flexibility for the grantees. Without a certain degree of a standardization, the other challenge of scale can't be addressed, which is how you aggregate results across a multitude of grants. That's been one of our most interesting challenges. Not just measuring impact, reporting on it, learning from it, but how do we do it in a way that allows us to change the narrative from reporting on one grant at a time? What's happening as a result of a portfolio of grants? It's a challenge - One person’s standardization is another person's reductionism.
"One person’s standardization is another person's reductionism."
There's a series of dance steps that you’re constantly doing to try to strike a workable balance.
Ben: That challenge is also such a big opportunity – The distinction of size versus breadth is important we have a huge breadth. We give around 700 grants a year, all in one Province.
Blair: Across 23 different grant results.
Ben: Sounds about right, ha. Maybe 24. And it’s that breadth that allows us to be that bridge between that local context, the grantees, where they’re at, and the need for rigor. I don't know of anyone else who has potential sample 40 different organizations working on getting kids physically active, measuring in the same way. I mean, the potential of a dataset like that to influence the field, to share what’s working in different contexts, to improve our internal decision-making, to share it with the sector, to co-create with the sector using the evidence that comes with this. It's an incredible opportunity. The huge sign of success would be once we start having that feedback loop.
Blair: The other measure of success for us is that more and more organizations, and people working in the field, are actually engaged in more meaningful impact measurement and reporting, and using that information. We always knew there was a huge appetite in the sector for being better at capturing results and knowing what was working and what wasn't. Sharing that kind of thing, to have an appetite to do something and to be actually able to do it. My experience over the last decade has been really encouraging. There's always resistance to new things. There is always a bit of fear of the unknown. There is always a bit of healthy questioning of that next latest, science-y, or academic approach to doing this work, but at the end of the day, I really think that there's been a growing shared value proposition around getting better at measuring impact and learning from it.
And to me, that's the ultimate success. We’re not all the way there yet, but I've been encouraged over the last number of years about how I think that that trajectory is increasing in slope.
Subscribe to our newsletter
Stay up to date with our latest news & stories