Nonprofit Chronicles

Journalism about nonprofit organizations and their impact

Hotels routinely survey guests after their stay. Airlines, too. Amazon wants to know whether I liked the book/TV show/kitchen gadget. Nonprofits? Not so much.

Businesses strive to measure–and improve–their performance, often by seeking feedback from their customers. Nonprofits and foundations lag behind, judging from a survey released last week by the Center for Effective Philanthropy.

“The typical nonprofit in the study allocates just two percent or less of its budget to assessing its performance, and few employ staff who are dedicated to this work full time,” the survey says.

anxiety-treatmentMeasuring performance can be scary. Hey, did you look forward to tests back in high school?

No nonprofit wants to admit failure. No donor to wants admit that money was wasted.

Yet those who overcome their anxiety to measure their impact will grow stronger.

Consider the Stages Repertory Theatre, an edgy regional theater in Houston. Stupid F—ing Bird, an adaptation of Chekhov’s The Seagull, just closed there and Bad Jews, a dark comedy about conflicts among grandchildren of a Holocaust survivor, just opened. This place is willing to take risks.

If you go to a show at Stages, you get an email the next day asking you about 20 questions, some predictable — your age, how you heard about the play, whether you found a place to park — and others a bit more thought-provoking, like

Overall, how strong was your emotional response to the performance?

and

At any point during the performance, did you lose track of time and become fully absorbed?

This is especially interesting because Stages, unlike most nonprofits, already has a valuable form of audience feedback: Ticket sales.

Lise Bohn, director of resource development at the theatre, says that Stages wanted more.

Really, how did you like the show?

But really, how did you like the show?

“We know who’s coming, when they’re coming, how much they pay, but that really doesn’t tell us very much, first, about what their experience is, and, equally important, what our impact is,” she told me by phone.

Stages aims to “enrich,” “stimulate” and “captivate” audiences; tickets don’t tell you much about that. So it turned to a consulting firm called WolfBrown, which has developed a customizable tool called Intrinsic Impact that arts groups like Stages can license to collect and analyze audience reactions.

Stages, which has been using the tool for about 18 months, is learning which shows captivate audiences, which one draw in diverse or young audiences and, of course, which fall flat. It won’t use audience feedback to tinker with with a play in the midst of a production, but it will use the results to tailor its marketing and select future performances. Stages encourages people to respond to the surveys by offering incentives, such as discounts on a future show.

There’s a whole lot of theory behind Intrinsic Impact, but what matters here is that Stages is making a real effort to measure something that’s hard to measure. “It makes a world of difference, just having some data at your fingerprints, even if it is not perfect or a huge data set,” Bohn says.

Exactly. Most nonprofits need not worry about Big Data. A little data will do.

Which brings us back to that survey from the Center for Effective Philanthropy, called “Assessing to Achieve High Performance: What Nonprofits are Doing and How Foundations Can Help.” (Try fitting that in a headline.)

Virtually all of the 183 nonprofits surveyed said they collect information about their impact. But this can mean almost anything. Many report on their “reach”–they count many meals they serve, or how many water taps they install in poor countries. These are outputs, not outcomes.

More telling was the survey finding that most nonprofits spend 2% or less of their budget measuring their performance, and that 60 percent have no full-time staff or third-party evaluators devoting to measuring performance.

“What’s the right number to be spending?” muses Ellie Buteau, vice president of research at the center. “We couldn’t find any benchmarks out there. Two percent seems quite low to us.”

To me, too. Think, after all, about the value of measuring performance. It should be used to revisit strategies and adjust tactics. It’s an excellent way to evaluate staff. (The compensation of managers of individual hotel properties is based, in part, on the results of customer surveys.) Nonprofits that can demonstrate high performance should better positioned to attract foundation support.

Foundations, too, need to bring a sharper focus to performance. They’ve known that for years. A 2013 survey by the same Center for Effective Philanthropy found that 51 percent of foundation CEOs said “grantees difficulty in assessing their progress” was one of the greatest barriers to foundations’ ability to make progress.

This new report concludes:

For nonprofits to be able to gather the most meaningful data, and use it for internal improvement — as well as to share with other organizations — they’ll need more support from foundations. They’ll need the financial support to increase the modest sums currently being spent on performance assessment. 

That’s a polite way to say it. The nonprofit sector, I’m learning, is nothing if not polite.

The fact is, foundations need to do more than support performance assessment. They need to insist that nonprofits measure their impact. Teachers don’t ask students if they want to take tests; they require them.

Foundations should do the same. Politely, of course.

5 thoughts on “Foundations, nonprofits and performance anxiety

  1. pbuchanan says:

    Thoughtful post, and we appreciate the coverage of the Center for Effective Philanthropy’s (CEP) new research. But the statement that nonprofits don’t survey their intended beneficiaries (“not so much”) is inaccurate.

    In fact, research we conducted — through the same grantee panel that informed the research discussed above — shows that almost all nonprofits in our sample (designed to be representative of grantees of larger foundations) do obtain beneficiary feedback. The overwhelming majority use surveys to do so. See that report here: http://www.effectivephilanthropy.org/wp-content/uploads/2014/10/CEP-Hearing-from-Those-We-Seek-to-Help.pdf

    This commitment is impressive given that getting feedback from beneficiaries is much more complicated for, say, an organization serving homeless youth than it is for an airline. (An aside: I travel a lot and, honestly, it’s not clear to me that the airlines surveying their passengers has resulted in an improved customer experience, so I don’t know why we’d look there for inspiration.)

    Performance measurement for nonprofits is uniquely challenging. There is no common unit of measurement, no analog to ROI or profit — so the comparison to measurement in business is not a helpful one.

    Further, the all too common view of nonprofits as lacking the will to undertake good performance assessment is likewise contradicted by our findings — both in the report you write about and in a 2012 report we released on the same topic. Nonprofit leaders are working to understand and improve their performance — that’s what our findings show. So the answer is not more foundation requirements; the answer is more resources and support for performance assessment that helps nonprofits do their work better.

    That’s what nonprofits want — and need. Nancy Csuti of the Colorado Health Foundation recently posted a very thoughtful and candid reflection on this topic on our blog. http://www.effectivephilanthropy.org/a-call-to-rethink-how-we-support-grantees/

    Final note: for those for looking for resources on nonprofit performance assessment, I suggest the Performance Imperative as a good place to start.

    http://leapofreason.org/performance-imperative/

    Thanks again for the post.

    Phil Buchanan
    President, The Center for Effective Philanthropy (CEP)

    Like

    1. Marc Gunther says:

      Thanks, Phil, for your response feedback. I wholeheartedly agree with two points you make. (1) Performance measurement for nonprofits is uniquely challenging. (2) We don’t want to look to the airlines for inspiration!

      I hope you’re right that most nonprofits collect feedback from their beneficiaries, learn from it and then work to improve their performance. I’m not so sure. If true, I’d like to see more nonprofits and foundations be transparent about how they measure their impact, admit failures as well as showcase successes and publicly share their insights. That said, as a newcomer to the sector, I probably should refrain from making sweeping statements about NGO “performance anxiety.”

      Like

      1. pbuchanan says:

        Thanks, Marc. Regarding hearing from beneficiaries: I know I’m right for the population we surveyed — those nonprofits (many of which are quite small) receiving funding from larger foundations ($5 M in grantmaking or more). And of course nonprofit leaders wish to hear from those they serve and want to do right by them. Why else would they be doing this work?

        That said, I agree regarding more transparency about assessment. And I definitely think really getting to the right metrics can be very, very difficult, as you point out in your post.

        But there are some great exemplars. I’ll name two nonprofits located right here in the Boston area, where I am. BELL and ROCA.

        http://www.experiencebell.org/our-results
        http://rocainc.org/what-we-do/proven-outcomes/

        And there are foundations that have modelled transparency in evaluation for decades, such as Robert Wood Johnson Foundation and the Wallace Foundation.

        http://www.rwjf.org/en/how-we-work/rel/tools-and-resources.html
        http://www.wallacefoundation.org/knowledge-center/Pages/default.aspx

        Like

  2. ed maibach says:

    Indeed! Wal-Mart become a global juggernaut by relentlessly focusing on relevant performance metrics. It’s high time for organizations in the non-profit sector to embrace this mentality.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: