Outcomes Measurement

As an MBA seeking employ in the nonprofit sector, I am somewhat of a rarity. That’s not to say that the nonprofit sector is completely devoid of MBAs, but there are a number of barriers that stop the average MBA from going into the nonprofit sector. Among them are lower pay, and lower perceived prestige—many of the conversations I’ve had with my MBA classmates involve assertions that nonprofits are, on the whole, well-meaning but poorly organized. And at the heart of this assertion is the claim that nonprofits don’t know how to measure their impact, and therefore are often “just throwing spaghetti at the wall and seeing what sticks.”

I have two primary assertions that I’d like to defend in this post:

·         Nonprofits will benefit from becoming better at measuring outcomes (not an earth shattering assertion).

·         For profits aren’t that great at measuring the very things they encourage nonprofits of measuring poorly.

Measurement of outcomes

First, it is important to understand the difference between outputs and outcomes. For the sake of discussion, I will define outputs as things you do and outcomes as the end results of things you do on things you care about.  Outputs sound like this: “120 books given to a local literacy organization;” “100,000 hours of volunteer work provided by our program.” Outputs are helpful tracking numbers, but they are like “inventory” numbers in a for-profit.  Outcomes, on the other hand, answer the question “what did my outputs do?” My 120 books and 100,000 volunteer hours helped move a cohort of students from 15% of students on grade level in reading to 29% (a 14% increase). My work with this workforce development program helped make a 20% in the 5-year recidivism rate of the participants involved.

This highlights a key point—outcomes come in different flavors (namely short, medium, and long-term). K-12 education has recently become a core passion area of mine, so I will take this example. Many education, youth development, and poverty alleviation groups seek to end poverty, both on the individual and generational levels. Here is a progression of metrics that they may engage in:

·         Outputs—Number of students enrolled in school/an after school program

·         Short-term outcomes—Changes in school attendance rates (which is abysmally low in many of the communities these organizations operate in); changes in test scores, reading comprehension, math ability, reasoning ability and critical thinking, etc.

·         Medium-term outcomes—High school graduation rates. (In my admittedly limited experience, it seems that few nonprofits measure and report these outcomes, with education being perhaps an exception; I’ve seen few do this well). I would even put college acceptance rates in this category.

·         Long-term outcomes—College graduation rates; number of students employed soon after college graduation; long-term earning potential of graduates.

·         Very-long-term outcomes—average net worth of area families; average earning potential/employment level of area families.

If your goal is to end cycles of poverty in an area, it is going to take a long time to see those effects come to fruition. But if you are measuring progress against a BHAG (Big, Hairy, Audacious Goal) like ending generational poverty, it is not enough to simply improve the test scores of 2^nd graders. Those second graders need to grow into high school graduates who thrive in college and find gainful employment. College attendance (and graduation) needs to become the norm in a low-income community, rather than an oddity. Only when these things happen is the mission truly being accomplished.

Why nonprofits benefit from measuring

Nonprofits benefit from measuring predominantly because the words “proven effective” are very effective in attracting the top two resources nonprofits need—talented people and money. Simply put, people are more attracted to working for a nonprofit that is effectively accomplishing its mission than one that simply doesn’t know how it’s doing. And people are much more willing to invest in such nonprofits.

Turns out it didn’t take much to support that assertion.

Why traditional business are actually not that great at this

My MBA friends often seem to believe in the superior measuring ability of for profits. Businesses are much clearer about their (economic) bottom line than nonprofits are about their (social) bottom line. But note two things:

1)      Nonprofits are actually pretty good at measuring their economic bottom lines as well (I say this facetiously, as they submit yearly audited financial statements, so they are equally as good as their for profit counterparts).

2)      For profits seem to do a poor job measuring anything but short-term economic outcomes—namely profit, revenue, and market share (market share could be argued as medium term, depending on the context). The fact of the matter is that corporations, even ones with great CSR departments, do not seem to have an overall handle of the environmental and social effects they create in any given year. And they definitely don’t track the long-term effect they have on the earning potential of an area, or on poverty, or literacy, or anything of the sort. In fact, many businesses don’t even measure their own market share—they typically buy data from companies like Nielsen or Kantar to do that.

The fact of the matter is that social impact is much harder to measure than profit. That does not excuse nonprofits from measuring, but is simply to say that neither sector is good at social measurement yet.

The moral of the story

This world needs top talent in highly effective nonprofits. Nonprofits need to measure their outcomes, not just their outputs, to figure out what is working and to draw that talent (and the resources to pay them) into the organization. MBAs need to realize that measuring profit is easy, and that if their company needed to measure social impact, it would be pretty horrific at it. And everyone needs to start identifying what their long- and very-long term social impact metrics might look like, and start working towards those outcomes.

Photo from www.ouropendoors.com