TechSoup.org The place for nonprofits, charities, and libraries

Defining and Measuring Outcomes for basic human needs agencies

Defining and Measuring Outcomes for basic human needs agencies

  • United Ways throughout the country are pushing to have agencies they fund measure not only their "outputs", people served, food provided, clothes provided, funds provided for RX, rent, gasoline, etc - - but also to define and measure the "outcomes" of such services.

    Anyone other there associated with a "basic human needs" type agency, providing assistance in the types of items listed above, who can provide input on what outcomes, and indicators they measure and report?

    Bob Alston

    Free case management software for food pantries/emergency services 501(c)(3) organizations. See: http://webpages.charter.net/bobalston/bob1.htm

  • Bob,
    There is a wealth of information and examples of outcomes measurement at United Way's national website:
    http://national.unitedway.org/outcomes/resources/
    Steve Capistrant scapistrant@symphonyinfo.com Symphony Information Services www.symphonyinfo.com
  • Thanks Steve. Lots of info but as best I could tell, only one document contains sample program outcomes - not available online. I will order it.

    Anyone else know of any resource available on-line that lists outcomes and indicators for basic human needs organizations? Surely someone is already doing this??

    Bob Alston

    Free case management software for food pantries/emergency services 501(c)(3) organizations. See: http://webpages.charter.net/bobalston/bob1.htm

  • I bought the United Way of America book. It defines outcomes as "Outcomes are benefits for participants during or after their involvement with a program. Outcomes may relate to knowledge, skills, attitudes, values, behavior, condition, or status. Examples of outcomes include greater knowledge of nutritional needs, improved reading skills, more effective responses to conflict, getting a job, and having greater financial stability."

    I am a volunteer with a local United Way supported agency that provides emergency food, funds (avoid utility shutoff, avoid eviction, etc). A consultant has completed 10 months of outcomes training and has defined the short term outcome as "confirm that you are serving the right population and denying services to those who may come to you but who are not the intended population". This just doesn't seem like a real outcome to me.

    Your assessment?

    Bob

    Free case management software for food pantries/emergency services 501(c)(3) organizations. See: http://webpages.charter.net/bobalston/bob1.htm

  • Bob,

    I will respond from the perspective of 1) a former United Way staffer who trained agencies on defining and measuring outcomes, and 2) a software developer focused on social service providers.

    You are justified in feeling underwhelmed by the consultant's suggested outcome. Technically, however, that outcome is correct and leads to more efficient service delivery. But it is only a short term outcome, and needs to be supplemented by an outcome that tells a "story", some kind of life impact result that inspires your funders to keep giving. So your consultant's conclusion is accurate but incomplete.

    You can define some good long-term outcomes for your program, but don't anticipate being able to measure them accurately. And that's OK. A good outcome in your case might be "Children in economically distressed households will have more stability and long-term success in their life due to fewer disruptive moves out of schools and neighborhoods".

    But the elegance of the United Way model for outcome measurement starts to break down with "basic services", and especially "emergency services." The target population is extremely transient, they lack phones and cars, and are too stressed to prioritize follow ups with care providers. It is unlikely that you'll find a way to measure this outcome that is statistically valid.

    The good news is that a significant number of funders (albeit not all) understand the problem and are willing to take the leap of faith required to accept the outcome. That is, they will accept the outcomes minus the indicators. But to hedge your bets, make sure to also report on raw outputs (headcounts, service counts) as well.

    Our organization publishes a document that provides a more detailed discussion about outcome/indicator measurement challenges like the one you have identified. It is written with reference to a specific software package we support, but the generic concepts are useful in any case.

    Last, I too have looked around for a nice, idea-provoking reference of outcomes and indicators but have come up empty-handed. I'd love to hear from anyone else about good resources. For the moment, it looks like we will have to do without a template.
    Steve Capistrant scapistrant@symphonyinfo.com Symphony Information Services www.symphonyinfo.com
  • Thanks.

    It seems to me that what the consultant defined as an outcome is really a measurement of something. I believe the outcome of providing emergency food for x weeks is "that at risk people have nourshing food for x weeks". In this case the output is almost exactly the same as the outcome. Further to me it seems that all the consultant has done is to suggest measuring the output in another way. The agency already provides statistics on persons served by zipcode, age group, ethnicity and gender.

    The best sources I have found for definitions of outcomes for emergency services such as this are in:

    1) The United Way's "Measuring Program OUtcomes: A Practical Approach" which is $5.00 plus shipping and can be ordered here:
    United Way book link[

    The web page I referenced above has the Outcome definition I quoted in an earlier post in this thread. Also reference pages: 20 and 76 ("The assistance is very short term").

    2)Outcome link2


    Q5. What about situations where clients outcomes and agency outcomes are different? E.g., a client may want to eat a meal . The agency may want to increase the skills and tools to make the client more self-sufficient.
    .
    .
    In the example you cite, if the mission of the organization is to increase self-sufficiency of individuals that are experiencing hunger, an appropriate initial outcome meeting participants' immediate need would be that participants have adequate food for a defined period of time.

    3)Output link 3

    4)Output link 4

    5)
    Output Link 5

    Would you agree with my assertions above?

    Bob

    Free case management software for food pantries/emergency services 501(c)(3) organizations. See: http://webpages.charter.net/bobalston/bob1.htm

  • Bob,

    I agree, people often get sloppy with how they define outcomes, often confusing them with outputs, or even inputs. In my mind, the commonly used terms are:

    1. Inputs: the resources you invest (time, actions, money)
    2. Outputs: the raw counts of people served and services provided.
    3. Outcomes: general statements of how the world will be changed (directional, not statistical)
    4. Indicators: Statistically measurable methods for evaluating if something related to an is accomplished.

    Your make an interesting point about the difference between client outcomes (e.g. client will get a meal) and agency outcomes (client will receive skills needed to be self sufficient). Ultimately, nonprofits are funded to produce CLIENT outcomes. But AGENCY outcomes are often needed for two reasons:

    1. As a proxy for a client outcome, when the client outcome is not measurable. For example, if you can't realistically measure "Families won't skip meals due to lack of income", you might have to measure "Families report that our services have provided improved means to increase or stabilize their income".

    2. As an intermediate outcome. Sometimes the chain of steps toward reaching an ultimate result is so extended and/or complex that you need to measure intermediate steps as well as ultimate results. An intermediate outcome would look like "Our agency will provide clients with information and skills to increase their self sufficiency".

    These same challenges exist at the indicator level. You sometimes need proxy indicators and intermediate indicators.
    Steve Capistrant scapistrant@symphonyinfo.com Symphony Information Services www.symphonyinfo.com
  • I have collected the best references I could fine on defining outcomes for food pantry, emergency assistance agencies and posted on my web site:

    http://members.cox.net/tulsaalstons/Outcomes.htm

    If anyone knows of other good references that I should add, please let me know.

    Bob Alston
    bobalston9@yahoo.com

    Free case management software for food pantries/emergency services 501(c)(3) organizations. See: http://webpages.charter.net/bobalston/bob1.htm

  • "Anyone other there associated with a "basic human needs" type agency, providing assistance in the types of items listed above, who can provide input on what outcomes, and indicators they measure and report?"

    There are probably not many agencies providing services that address "basic human needs" on TechSoup, unfortunately -- so have a look in your own community, and ask if any of these agencies would be willing to share with you the way they measure "outcomes." Also, your nearest nonprofit development center may be able to help -- I believe there a list of such in the Nonprofit FAQs.

    Organizations in the developing world have long been asked to provide these type of outcomes. Might be worth visiting the Development Gateway and United Nations web sites to see what's out there from other countries regarding such measurements.
  • As part of the Homeless Management Information System initative, HUD (Housing and Urban Development) has begun the process of incorporating outcomes measurement into all of its contracts with homeless service providers.

    A search of HUD's website will provide you with some documentation on the effort: www.hud.gov
    -------------------------- Paul Hebblethwaite www.slhandph.com/nonprofit --------------------------
  • We are wondering if this software can be shared by multiple agencies so they can share the data among them?

    Sue