Tag Archives: Patrick Germain

Making Evaluation Happen When you are not in the Room: Part 1, Starting From Scratch [Guest post by Patrick Germain]

This is the first of a three part series on how internal evaluators can think about building their organization’s evaluation capacity and sustainability and is based on a talk at Eval13 by the same name.

strategy

Any evaluator, internal or external, working to incorporate evaluative practices into nonprofit organizations must engage a wide variety of staff and systems in the design, implementation, and management of those practices.  The success of those efforts will be decided to a large extent by how non-evaluators are brought into the evaluation tent, and how evaluation is integrated into administrative and service delivery systems.  But how do we even begin?

Starting from Scratch

There are three main steps to coming up with any kind of strategy, including a strategy to build evaluation capacity.

1)   Understand the context

Without knowing where you are starting, it is very hard to set realistic goals.  So before you even start on your journey to build evaluation capacity, you have to know what you are working with.  Get to know the people you will be working with, the restraints and requirements, the values and priorities of the organization.  Conduct a SWOT analysis.  Determine who your allies will be, where your largest barriers will arise.  What will the culture of the organization support, and what is anathema to it?  Much like a body will reject any transplant that is incompatible with it, an organization will respond poorly to an intervention that doesn’t resonate with its culture.

2)   Define your destination and your path

Saying you want to ‘build evaluation capacity’ is not a good enough goal.  What does that mean?  What does that even look like? And how are you going to get there? What are interim benchmarks you can use to determine progress?

I have found three general strategies that have worked well for me: (1) make sure leadership is setting clear expectations for staff participation in evaluation activities, and holding them accountable for it, (2) start working with the high performers and people who already ‘get’ evaluation to create easy wins and visible successes, and (3) focus on the priorities of the people with influence – by convincing them of the value of evaluation, they will begin to shift the center of gravity in the organization closer to evaluation values.

3)   Prepare the foundation

What is the bare minimum in resource needs for you to accomplish your goal?  (Hopefully you were clear about resource needs before you even took the job.)  This is going to be different for every situation, but we probably all know the feeling of not having enough resources to accomplish our goals.

For me, these things recently included: technology, training for evaluation staff, time commitment from people throughout the organization, and coworkers who would support me if I got backed into a corner.  Some of these things I had to get budgetary approval for, but most of them were more about building strong and trusting relationships.  I had to be transparent about my intentions and manage everyone’s expectations about what they were expected to give, and what they could expect to get from working with me.  The first couple of months were more about creating strong relationships than about doing any ‘real’ evaluation work.

Patrick Germain

Patrick Germain

What strategies have worked for you?  What have your pitfalls been when starting a new capacity building effort?

Next post, I’ll discuss how to create momentum around evaluation capacity building efforts.

 

 

 

Patrick Germain is the Director of Strategy and Evaluation at Project Renewal, a large homeless services organization in New York City and is the President of the New York Consortium of Evaluators, the local AEA affiliate.

Getting the Resources You Need for Evaluation [Guest post by Patrick Germain]

Patrick Germain

Hi evaluators!  My name is Patrick Germain and I am the Director of Program Evaluation and Quality Assurance at Project Renewal, and the Founder and Director of Performance Management Professionals, a community of practice based in New York City on issues of Performance Measurement and Management and Evaluation.

Organizations are rife with politics, and politics is often the decisive factor in whether evaluations get used. This is the third post on how I navigate and influence organizational politics to ensure that evaluation is appropriately supported and used in decision making. Check out my earlier posts on Managing Your Evaluation Reputation and Preempting Conflict in Evaluation.

Week Three: Getting the Resources You Need for Evaluation

‘Doing more with less’ seems to be standard operating procedure in the non-profit sector these days. Need for social services continues to rise, and funding continues to be scarce – and in the rush to scrape up whatever funding there is, evaluation is often left in the dust.

So what can internal evaluators do to make sure we get the resources we need? Unfortunately, not a whole lot… But that doesn’t mean we have to throw up our hands.  It is what it is, but that isn’t how it has to be!

Even if you can’t magically double your budget, there are some things you can do to get your needs higher on the organization’s priority list:

  • Understand the priorities of the decision makers – know what they want, and demonstrate how you having additional resources will help achieve that goal.
  • Get lots of different people to ask – A brief illustration: From day one, I knew that our IT capacity was insufficient – not only for me to do my job, but also to solve a myriad of problems throughout the organization.  Every time someone highlighted a relevant problem, I explained how IT solutions might be able to fix it and asked them to help me build pressure for more IT capacity.  I cultivated the understanding that there was huge unmet potential that could be unleashed with more investment in IT.  At a certain point, it became a clearly articulated organizational need that the decision makers had to place higher on the priority list.
  • Don’t make it about yourself – make it about impact and efficiency.  Demonstrate the impact that your work will have on the clients or constituents. Show how it will help the staff and the programs.  Demonstrate that evaluation is an investment, and not just a cost.
  • Get evaluation built into future budgets – It is hard to shift around resources that are already allocated, but it’s a lot easier to get evaluation built into future funding streams.
  • Align with the strategic plan – My organization recently went through a strategic planning process, and it was my one goal in life to make sure evaluation got incorporated into it. (ok, so maybe that’s an exaggeration).  But our strategic plan now has initiatives directly relating to building evaluation capacity.  Evaluation is also integrated into measuring whether we actually achieved what we set out to do in our strategic plan.
  • Ask for it and be ready to justify it – It’s amazing how often people forget this one critical step.  If you don’t make it clear what you need and why, it is doubtful you will get it.  But remember, when you are justifying your request, frame it within the priorities of the decision makers.
  • Leverage other organizational resources – I was surprised at how many of our program directors were excited to participate in evaluative activities.  I asked for volunteers, I asked to borrow staff for a short period, I asked to borrow space and equipment, I asked for help on specific questions or challenges.  If you have done a good job at building your reputation and building allies, you might even get people to agree to it!

It is entirely possible that these ideas won’t work in your context.  Do any of these strategies apply to your position? Are there other ways you have gotten resources?

– Patrick Germain

P.S. Want to learn more? Look for me at the American Evaluation Association conference in Minneapolis, where I will be presenting on some of these issues.

Preempting Conflict in Evaluation [Guest post by Patrick Germain]

Patrick Germain

Hi evaluators!  My name is Patrick Germain and I am the Director of Program Evaluation and Quality Assurance at Project Renewal, and the Founder and Director of Performance Management Professionals, a community of practice based in New York City on issues of Performance Measurement and Management and Evaluation.

Organizations are rife with politics, and politics is often the decisive factor in whether evaluations get used. This is the second of three blog posts on how I navigate and influence organizational politics to ensure that evaluation is appropriately supported and used in decision making. You can read last week’s post here.

Week Two: Preempting Conflict in Evaluation

I was once told that I had an “excess of diplomacy.” Although my boss’s tone implied criticism, I took it as a compliment! In my role as an internal evaluator, diplomacy is a must. It takes a long time to gain the trust of program staff, and if you aren’t careful it can all vanish quicker than you can say “But correlation is not causation!”

Let me share a story of how I prepared for presenting my first evaluation to the organization’s senior leadership. I knew there were big risks and big opportunities in this first experience – so I dipped into my “excess of diplomacy” and put it to good use:

Long before any evaluation was conducted, my first goal was to train the managers to “jump to questions, not conclusions.” (“Good Stories Aren’t Enough” Martha Miles). I coached all staff to engage in a constructive conversation about performance data, and not jump to conclusions or cast blame.

Secondly, before discussing the findings in a large group, I sat down with the senior manager, went over the findings, and said “Here is what I see in the data, and if I were you, here are the questions I would ask …” Separately, I sat down with the program director and said “Here is what I see in the data, these are the kinds of questions I would ask if I were your boss, and here is how I might begin to answer them if I were you…”

In the meeting later that week, I asked the CEO to open the conversation with: “We are here to learn, not to judge. Remember, we should be jumping to questions, not to conclusions.” Then, throughout the next hour, the senior managers asked the kinds of questions I had helped them prepare, and the program manager provided the kinds of responses I had worked with them on! When someone got perilously close to jumping to conclusions the CEO would interject with “Let’s stick to constructive questions, please.”

At the end of the meeting, everyone walked away feeling like we had a fruitful conversation, and everyone thanked me for helping them prepare for the meeting. The CEO came up to me and congratulated me on how well the meeting went. As time went on and we continued to have similar conversations, people began to be less fearful or accusatory, and more inquisitive and engaged in the conversation.

Now, this worked for me because each of the individuals wanted to have a good experience with evaluation but didn’t quite know how to do it – I therefore became a resource for them to do their job, not another administrator making them jump through hoops. I made sure to set clear expectations, give the people the ability to meet those expectations, and asked the CEO to support me in appropriately guiding the conversation.

How do you manage conflict as it relates to evaluation?  What strategies have you found successful for building a supportive political context for evaluation?

– Patrick Germain

P.S. Want to learn more? Look for me at the American Evaluation Association conference in Minneapolis, where I will be presenting on some of these issues.

Managing Your Evaluation Reputation [Guest post by Patrick Germain]

Patrick Germain

Hi evaluators!  My name is Patrick Germain and I am the Director of Program Evaluation and Quality Assurance at Project Renewal, and the Founder and Director of Performance Management Professionals, a community of practice based in New York City on issues of Performance Measurement and Management and Evaluation.

Organizations are rife with politics, and politics is often the decisive factor in whether evaluations get used. This is the first of three blog posts on how I navigate and influence organizational politics to ensure that evaluation is appropriately supported and used in decision making.  

Week One: Managing Your Evaluation Reputation

When I started this job, I knew that my presence was likely to elicit wary skepticism from program staff – if not outright hostility. So I worked to position myself within the organization to avoid power struggles as much as possible:

Step 1: I clarified my role.

I worked with my team to define our department’s mission, and then broadcasted it as widely and frequently as possible:

Our role is to enhance the agency’s ability to achieve its mission by developing and facilitating processes and systems that measurably improve the quality and effectiveness of our services and operations, and to support agency leadership in managing towards high performance.

By framing my role as one of an internal consultant whose primary goal was to support the agency’s leadership, I was saying that I was there to help them achieve their goals, not just to question everything they were doing.

Step 2: I talked the talk – Now I had to prove it!

I set out to establish relevancy, credibility, and integrity:

  • The leadership’s priorities were my priorities – I got the right information to the right people at the right time, and in the right format.
  • I made sure that whatever I did was accurate, clear, and actionable.
  • I followed through with what I said, and stuck to my guns when I was asked to bend the rules ‘just this once.’

Step 3: I became a problem solver.

One of my least favorite sayings is “It is what it is.” I want to yell out: “But that isn’t what it has to be!”  I began to cast myself as the ever-optimistic champion of change, and made sure to get some easy wins under my belt. Anything I could do to solve a problem, I did, no matter how small. We got a faster internet connection, we got the boxes taken out of the back bathroom, we convinced our funder to change their targets, we got carrots for our meetings instead of cookies, we changed the printer defaults to do double sided printing. And on and on. Not only did this win me allies, but people started to understand that we had the power to change the status quo.

Now, people knew that my goal was to help them achieve their goals and that I was trustworthy. They knew that I could help them figure out solutions to their problems and that I was able to achieve results. While none of this was directly about evaluation, these efforts positioned me so that by the time I got around to my evaluations being used, I had created an environment in which I was more likely to be listened to and trusted.

What has worked for you? What strategies have you used position yourself in your organization?

– Patrick Germain

P.S. Want to learn more? Look for me at the American Evaluation Association conference in Minneapolis next month, where I will be presenting on some of these issues.