Tag Archives: evaluation capacity building

Making Evaluation Happen When you are not in the Room: Part 1, Starting From Scratch [Guest post by Patrick Germain]

This is the first of a three part series on how internal evaluators can think about building their organization’s evaluation capacity and sustainability and is based on a talk at Eval13 by the same name.

strategy

Any evaluator, internal or external, working to incorporate evaluative practices into nonprofit organizations must engage a wide variety of staff and systems in the design, implementation, and management of those practices.  The success of those efforts will be decided to a large extent by how non-evaluators are brought into the evaluation tent, and how evaluation is integrated into administrative and service delivery systems.  But how do we even begin?

Starting from Scratch

There are three main steps to coming up with any kind of strategy, including a strategy to build evaluation capacity.

1)   Understand the context

Without knowing where you are starting, it is very hard to set realistic goals.  So before you even start on your journey to build evaluation capacity, you have to know what you are working with.  Get to know the people you will be working with, the restraints and requirements, the values and priorities of the organization.  Conduct a SWOT analysis.  Determine who your allies will be, where your largest barriers will arise.  What will the culture of the organization support, and what is anathema to it?  Much like a body will reject any transplant that is incompatible with it, an organization will respond poorly to an intervention that doesn’t resonate with its culture.

2)   Define your destination and your path

Saying you want to ‘build evaluation capacity’ is not a good enough goal.  What does that mean?  What does that even look like? And how are you going to get there? What are interim benchmarks you can use to determine progress?

I have found three general strategies that have worked well for me: (1) make sure leadership is setting clear expectations for staff participation in evaluation activities, and holding them accountable for it, (2) start working with the high performers and people who already ‘get’ evaluation to create easy wins and visible successes, and (3) focus on the priorities of the people with influence – by convincing them of the value of evaluation, they will begin to shift the center of gravity in the organization closer to evaluation values.

3)   Prepare the foundation

What is the bare minimum in resource needs for you to accomplish your goal?  (Hopefully you were clear about resource needs before you even took the job.)  This is going to be different for every situation, but we probably all know the feeling of not having enough resources to accomplish our goals.

For me, these things recently included: technology, training for evaluation staff, time commitment from people throughout the organization, and coworkers who would support me if I got backed into a corner.  Some of these things I had to get budgetary approval for, but most of them were more about building strong and trusting relationships.  I had to be transparent about my intentions and manage everyone’s expectations about what they were expected to give, and what they could expect to get from working with me.  The first couple of months were more about creating strong relationships than about doing any ‘real’ evaluation work.

Patrick Germain

Patrick Germain

What strategies have worked for you?  What have your pitfalls been when starting a new capacity building effort?

Next post, I’ll discuss how to create momentum around evaluation capacity building efforts.

 

 

 

Patrick Germain is the Director of Strategy and Evaluation at Project Renewal, a large homeless services organization in New York City and is the President of the New York Consortium of Evaluators, the local AEA affiliate.

Exploring the Non-Profit Paradox – Evaluation and Non-Profits [Guest post by Jamie Clearfield]

jamie_clearfield

Jamie Clearfield

Hello, I’m Jamie Clearfield from the Collaborative for Evaluation and Assessment Capacity (CEAC) at the University of Pittsburgh.

Having had the privilege to work with a range of community-based organizations (CBOs) and non-profit organizations in the United States and internationally, I have been continually struck by the ingenuity of organizations to work towards program goals, as well as a lack of understanding of the role of evaluation in the work.

I get it – small community organizations and non-profits are often (if not nearly always) in the fight for their survival – another crisis is always at the end of another email, phone call, or text message. How many times was I working in another country when I would get a text message from the director of a school I work with that the feeding program funding has been cut, we were about to be evicted, or a program partner decided to “go in a different direction”? Incorporating evaluation into the existing chaos – particularly when much of the program staff is unsure of what evaluation is, how it is conducted, or what it can mean for the organization – can be a difficult sell, even to the most innovative of organizations. Yet herein lies the paradox for small CBOs and organizations – evaluation can help avoid the ongoing crisis cycle and can help organizations plan and prepare for the days ahead, while also identifying where programs are succeeding or need to be rethought.

For those working in evaluation, this paradox offers many challenges and opportunities to work with CBOs and non-profits, helping them strengthen and expand their programming to a wider audience. The challenges however are many – among a few are how to reach small organizations that may not understand the need of your services, who may not be able to pay market rates for evaluation, and who may have limited human and social capital to make evaluation a worthwhile endeavor. There is also the challenge of dependency – small organizations may come to rely too heavily on the evaluator to conduct the evaluation year after year, regardless of whether it is the best use of the organization’s or evaluator’s time or resources. Dependency results from a gross misunderstanding of the role of evaluation and whom evaluation really should benefit.

So what are some solutions? There are no zero sum answers to these questions – it is an ongoing trial and error experiment. At the heart of many of these questions is the need to develop strong working relationships between evaluators and CBOs/non-profits. Doing so not only will allow for more candid honest working relationships in the long run, it can also help CBOs/non-profits understand that they are not alone, and that working with an outside entity (an evaluator) can help develop their community. Evaluations firms/evaluators can open the dialogue with CBOs/Non-profits by meeting organizations on their own terms – explaining the role of evaluation in terminology that works for the organization and mapping out, with concrete examples, how evaluation can benefit the organization. It is not merely a question of marketing your services, it is developing a lasting relationship. This may involve attending events hosted by the CBO, organizing meetings with staff, etc. It can be a slow process, however worthwhile.

Thoughts – how can evaluators form better relationships with CBOs/non-profits that result in stronger, more useful evaluations?