Standards Future Series

The Future of Impact Reporting: Professor Mark Fox at the University of Toronto talks the trajectory of individuals, the need for privacy legislation, and why he focuses on cities

Posted By SAMETRICA Customer Success on May 15, 2018 2:26:23 PM

Professor Mark Fox at the University of TorontoDr. Mark Fox is a Professor of Industrial Engineering and Computer Science where his current research applies Artificial Intelligence to Smart Cities. He received his BSc in Computer Science from the University of Toronto in 1975 and his PhD in Computer Science from Carnegie Mellon University in 1983, where he was an Associate Professor (tenured) of Computer Science and Robotics. He was a founding member of Carnegie-Mellon’s Robotics Institute. From 1981 through 1987 he founded and led the Robotics Institute’s Intelligent Systems Laboratory, and from 1987 through 1991 he co-founded and led the Center for Integrated Manufacturing Decision Systems. In 1993 Dr. Fox co-founded Novator Systems Ltd., a pioneer in out-sourced eRetail services and software. In 1984 he co-founded Carnegie Group Inc., one of the first companies to apply Artificial Intelligence to solving engineering, manufacturing, and telecommunications problems. He is a Fellow of the American Association for Artificial Intelligence (AAAI) and the Engineering Institute of Canada.


Tell us about you. What motivates you? 

I focus on building a society that's equitable, and civilized, meaning that it looks out for the wellbeing of people, that it provides them with their needs. That underlies everything that I do. Then you take that down to the next level - Where do people live? Well, the majority of people work within cities. You want to solve problems – You do it within the cities themselves.

Why the focus on cities? There are other units of measurement, that are also organized around institutions and people. The most obvious ones are the federal and provincial organizational unit. Why focus on cities specifically?

Well, for Canada the 80%, of the population lives in cities, 20%, of population does not. So if you want to feel the largest impact you focus on cities. I've told people at the provincial level that their role in society is to manage the empty space between the cities, and it’s the purpose of cities to manage the city itself. I stand by that characterization.

"I've told people at the provincial level that their role in society is to manage the empty space between the cities, and it’s the purpose of cities to manage the city itself. I stand by that characterization."

Ha. No one mincing words there. What are your professional interests?

I invest time in the Center for Social Services Engineering [at the University of Toronto] which is focusing on the extension application of engineering science, industrial engineering etc. to dealing with designing and delivering services to people who are in need. There, we deal with questions like, “How you take the 60,000 NGOs in Ontario and turn them into a single virtual NGO, so that they eventually utilize their resources within this place?”

We have other projects dealing with immigrant skills and transformation of those skills. We are building an ontology representing social services. We have spent time in the slums of Mumbai both in Dharavi and Santa Cruz, looking at the design of toilet blocks, how best to set up this physical infrastructure, and also the community support operations. There's a number of other projects that are underway. Ultimately, we deal with the whole area of how engineering contributes to the wellbeing of people.

And then part of my life is concerned with how you represent the knowledge that a city uses to provide its services. It's all about knowledge representation. It is focused on how to measure the performance of cities using these knowledge representations as a basis of deriving their performance indicators. We are concerned with how to analyze all this data to determine root causes of the for good performance of the city across different themes of sewage transportation, governance, economics, urban and shelter. 

What elements of today's impact reporting do you think will survive and thrive in the future? What do you think the future of impact reporting will look like?

 Well, here's the thing about impact reporting. Let’s make explicit what everybody already knows - That for whatever reasons we shy away from truly tracking the trajectory of individuals. So, whether it's privacy concerns, or lack of technology, or information gathering limitations, or integration of data from various services, our ability to track individuals and understand the impact of service delivery, is limited.

So, when we look at a lot of the data we gather today it's really oriented towards the service provider. We're spending a lot of time tracking the impact of that money. So, we ask agencies to track the impact. The only problem is that an agency will track measures that show the best light of that agency. Okay, but what we really need is metrics on the individuals themselves. We need to be able to track the individual over an expanded, extended period of time. We need to know their pathway through life, we need to know what interventions have taken place along that pathway through life, and we need to be able to see which pathways are effective, which interventions are not effective and then - Only then can you then take it back to the service providers and ask the question, you know, are the service providers effective? Now it may be the case that service providers are effective, but it's just the service they're providing that is the issue. Okay, so there's a distinction between the service we are providing and the service provider. The service provider may be very effective, except that the service has no impact whatsoever on individual.

So, in some sense we're measuring the wrong thing. The reason why we're measuring the wrong thing is because we're measuring the thing we can measure.

The future for me is when you're tracking individuals, their impact, successes, and failures and then tie that back to service provision.

How would that be achieved? How would we go to the level of tracking individuals? What are the roles that are going to allow that to happen? 

It's a good question. The very first thing you have to ask is, ”Why can we not do it?” The answer always goes back to privacy. There are cultural attitudes that are embedded in the institutions, whether its privacy reasons or whatever, that they don't want to provide the data that we need, so we have those barriers. The culture of organizations does not allow for the sharing of that information. We also have a lack of privacy legislation, that stop us from getting that information.

You know, can we put in a universal identifier so that for every service somebody uses, they just provide it? Well, you know what? The Ontario Health Insurance Plan (OHIP) card already exists, the health card already exists, so what more do we need? Is it hard to be able to track the information when they go to a shelter, or they go for food shelters or whatever? It's all doable, but it all starts with the desire to track the right information, that's really the issue - That desire to track the right information with the right privacy legislation, that’s the system’s current problems.

Yeah, it would be unacceptable to start from a place of wanting to demonstrate and manage outcomes, and then instead go to the dark places that we’ve seen social media organizations go to recently. Not only have they stomped on the idea of meaningful consent, they’re not even doing it for social good. They just want to sell services and products.  The spirit and the implementation of what you're suggesting must be different to be effective. 

So, first you have to recognize one thing. Regardless of what access you want to that information, it is being aggregated and acted upon by public and private organizations as we speak…I always tell people that if you are concerned about privacy of information, you are about fifteen years too late. Marketing companies have been gathering information about individuals for a long time and I use to be amazed at the type of targeting they were able to do in the year 2000, not even the year 2017, so they are so far ahead of what people realize is going on, it’s too late from a privacy point of view.  

"We're measuring the wrong thing. The reason why we're measuring the wrong thing is because we're measuring the thing we can measure."

So, you are not going to stop people from aggregating, we leave digital traces everywhere and then people are vacuuming it up, hoovering up those traces as we speak. 

So, given that, the issue is not whether people or individuals are going to aggregate the information. That’s a given. The issue is how to control it. The application of that information for the good of society - That is where the role of government comes in, in terms of passing laws and so if a company vacuums up a whole bunch of information that is detrimental to society, then those people should be in jail. It is no different to me then committing bodily harm, or some other serious crime where the punishment is to go to jail. I had hoped that the Canadian or North American government would look at the categories of information that you can aggregate, and which ones you cannot.  

We know one category right from the beginning. We know that identity, should be and is illegal. So that’s clear, but there’s lots that are not. The example of the Chinese government withholding or providing goods or services (paywall) based upon alignment with the government’s position – That is unacceptable to us. We have to figure out a way to put it to law that you cannot do that. The sad thing is that whatever you put into law, there’s always going to be people who break the law. [When someone breaks the law] the question should not be if someone who breaks a serious law is going to jail, but when are they going to jail. So, we need that legal system to support civil society the way it should.

Do you have any examples of precursors of this type of approach, that provide a proof point that this could potentially work?

Yeah, there are examples in the financial industry. The security commission has watchdogs looking at patterns of trade that are indicative of illegal trades. So, the question then becomes, can we create something that analyses digital traces to infer whether illegal uses of information are occurring? But we are so early in that, even in the financial sector. See the interesting thing about the financial sector is that the only “digital traces” that you need is the trades on the stock exchange, so it’s just a single entity that you’re watching as opposed to a million servers spread across the country, and so, you know, it’s always easier when you can narrowly define the scope of your problem. You know, “find the illegal trades in this particular market”. Um, but when we expand beyond that, it becomes a little more, or a lot more complicated.  

And so, in the financial world, the information you need is organized. The SEC requires filings on a quarterly basis. This is a whole reporting machine. For social stock exchanges, it’s not just financial reporting. There’s a whole issue of outcomes and that kind of measurement becomes really important.

"If you are concerned about privacy of information, you are about fifteen years too late"

Based on your experience in the market, what kind of timeframe are we looking at for these types of change?

When are outcomes going to be publicly available? They will become publicly available when the funding agencies think they should become publicly available. So, the question is, what motivates a company to publish their outcomes? The reason needs to lie in the demand part of the equation, the funder, or in competition for funding.  

The first one we understand. For example, Ontario Trillium Foundation says “these are the outcomes we’re going to measure” and you have to report on it, and that’s what they’ve done. Second, there could be demands on the part of the market, so on the part of social marketplaces like stock exchanges. That market has to define what the reporting requirements are on the part of the market. So those are the reasons why somebody would do it.

What is the timeframe for it? Well, we know that Trillum has already imposed outcomes-based reporting. We know that the Province [of Ontario] is interested in this reporting although those wheels turn more slowly. I don’t think we’re at a point yet where we have enough of a market where companies would publish that information in their prospectus. It really boils down to the prospectus so that potential investors see the information. And I, I just don’t know where we are on that. I have just never seen any social good company prospectus having their outcomes in that document.

So, how long is it going to take? It’s happening now. But it’s limited to Trillium-like funders. We know that there is a huge movement towards measurement, measuring outcomes and so I expect that over the next five years, more of these organizations are going to fall in line in terms of reporting outcomes. But the biggest problem that those organizations face is it creates an increasing administrative burden. When you're dealing with charitable organizations where in theory the administrative burden should be at let’s say 20%, 25% and we're now asking them the report even more stuff, it is going to drive up the percentage [of spend]. And that is, you know the schizophrenia that exists, we want to put the money into service delivery, and not into more administrative tasks, so in the end it becomes how quickly can organizations absorb the cost, so on and so forth.

Right. So, last question what role do you see yourself playing in creating that future?

Well, we firmly believe that standards need to exist in the area of the data that, we are reporting, and standards need to exist in the measurements that we perform based upon the data that we are representing. To introduce metrics, or indicators, independent of any standardization of the underlying data, it means that the measurement, the indicators that are reported are potentially incomparable because it could be created from different data.

The example that I use in the area of homelessness, is that you could be measuring shelter beds and you want to measure how well Toronto does in terms of providing shelter. You measure the number of beds, you measure the percentage of those beds that are occupied, and we compare ourselves to Montreal. Are we doing better? But, Montreal may or may not include shelters for battered women. We may or may not include shelters for family? Are they part of that or not?

Without the understanding of the standardization of both the representation of data and the definition of indicators, then we have no way of no way of really, honestly comparing the data.

So, what’s the contribution we are trying to make? We are trying to contribute to the standardization of the data representation. ‘Semantic interoperability’ is how we refer to it, and also the standardization of the indicators itself, we’re not in the business of defining the right indicators, but the people who are in the world of social work or public health, um, are in the business of defining what the indicators should be. We are in the business of translating those into computational representations that can be consistently applied across data that is being imported.


As told to Anshula Chowdhury exclusively for SAMETRICA. Transcript edited and condensed for clarity. 


Post a comment

Related Posts

Read More
Standards Future Series

The Future of Impact Reporting: Blair Dimock and Ben McNamee at Ontario Trillium Foundation

Read More
Press releases Standards

SAMETRICA software accredited by Social Value International

Read More
Standards Future Series

The Future of Impact Reporting: Professor Mark Fox at the University of Toronto talks the trajectory of individuals, the need for privacy legislation, and why he focuses on cities

Want to read more industry leading posts?

View All Posts