Tag Archives: evaluation resources

The Conference is Over, Now What? Professional Development for Novice Evaluators

At the American Evaluation Association’s annual conference in October 2013, I led a roundtable titled “The Conference is Over, Now What? Professional Development for Novice Evaluators.” We discussed ways that novices can deepen their knowledge, build their skills, socialize with other evaluators, and get involved in leadership positions. I compiled the notes here so more people can benefit from these resources.

Here are the best resources for novice evaluators:

aea365

aea365This is the American Evaluation Association’s daily blog located at aea365.org. You can read about everything from item response theory to slide design. Confession: I rarely read an entire post. Instead, I’m skimming the posts just to see the title, author, author’s organization, and the main gist of the content. This is a great way to stay up-to-date on the biggest trends in the field.

You should seriously write for aea365, probably 2-3 times a year, even if you’re new to the field. Just make sure you follow the contribution guidelines.

Affiliates and other organizations

AEA is the national-level mothership and there are more than 20 local and regional affiliates. You can find a full listing of affiliates here: http://www.eval.org/p/cm/ld/fid=12. Every affiliate is different. For example, the Washington Evaluators hold monthly brown bags, quarterly happy hours, and an annual holiday party. The Eastern Evaluation Research Society holds an annual 3-day conference. Other affiliates hold virtual book clubs, maintain blogs, or simply hold member meetings via teleconference.

You should join your affiliate. Seriously. The mailing lists are little nuggets of gold and worth every penny of that $25/year membership. The Washington Evaluators, for example, send job announcements almost every day, so you’ll always know which organizations are hiring and expanding. Don’t forget to attend the affiliate events too. (Sometimes people just pay dues but skip all the events, and then they don’t know why they’re not meeting anyone? This confuses me.) After a year, start planning small events yourself, like a brown bag. Then, join the Board.

Here are some additional reasons to join affiliates and ideas for getting involved:

data_community_dcThere are tons of additional evaluation groups. For example, the Environmental Evaluators Network, led by Matt Keene, holds forums for evaluators interested in environmental issues. If you’re in Washington, DC, the Aspen Institute holds quarterly breakfast panels focused on advocacy evaluation. At Innovation Network, we hold Ask an Evaluator sessions for nonprofit leaders. Tony Fujs and I also attend Data Science DCData Visualization DC, and Data Community DC monthly meetups. No matter your city, there are probably lots of events that fit your interests.

Blogs

First, check out evalcentral.com, run by Chris Lysy. Chris pulls in feeds from 60+ evaluation blogs so you’ll get exposed to a diverse set of perspectives. Chris even developed a daily email digest, so you can subscribe once to all 60+ blogs rather than monitoring your subscriptions to all the individual blogs. I suggest setting EvalCentral as one of your homepage tabs (along with your other must-haves like Gmail and Pandora) so it’s there every time you log into your computer. And again, I rarely read an entire blog post but I skim everything for the title, author, and main gist of what they’re talking about.

Second, check out AEA’s listing of evaluators and evaluation organizations who blog: http://www.eval.org/p/cm/ld/fid=71

I started blogging after watching Chris Lysy’s Ignite presentation at the 2011 AEA conference. Here’s Chris’ Ignite, which outlines just a few of the infinite reasons why evaluators should blog:

Coffee Break webinars

Coffee Break webinars are just 20 minutes long, so they’re a perfect way to squeeze in some quick professional development in the middle of a busy work day. The best part? They’re free for AEA members. I like to sign up for topics that I know nothing about. After 20 minutes, I’m not an expert, but at least I’ve got a basic understanding of that flavor of evaluation.

Conferences

Evaluation conferences include:

Do you know of additional evaluation conferences? Please link to them in the comments section below.

I also like to attend non-evaluation conferences to hear how non-evaluators are describing our work (they have completely different lingo and tend to value qualitative data way more than evaluators do).

eStudies

An eStudy is a 3- to 6-hour webinar run by AEA. eStudies are like mini grad school courses because they go in-depth on a particular topic (as opposed to 20-minute Coffee Break webinars, which just provide an overview of a topic). eStudies are broken into 90-minute chunks and there’s typically a homework assignment between each segment to help you practice your new skills.

For example, I participated in an eStudy about nonparametric statistics in which the instructor covered about 20 different nonparametric statistics, when to use each one, and how to perform the calculations in SPSS. We even got to keep her slides, which were full of step-by-step SPSS screenshots. Almost two years later, I still pull out my eStudy notes whenever I need to use some nonparametric statistics.

Journals

AEA offers two journals, the American Journal of Evaluation and New Directions for Evaluation. Both of these journals are included with your AEA membership. What a steal!

LinkedIn

These days, I can’t imagine an employer not doing a full internet search on new applicants. Make sure your LinkedIn profile has, at the bare minimum, a professional photo, your full work history (including dates), and your education history. You can also use LinkedIn to build your online portfolio (e.g., embedded slideshows from recent conference presentations, links to publications and projects, and your list of certifications).

Want to connect with other evaluators? Some awesome evaluation groups on LinkedIn include:

Do you know of additional evaluation groups on LinkedIn? Share your suggestion in the comments below. Thanks!

Listservs, mailing lists, and newsletters

First, check out EvalTalk: https://listserv.ua.edu/archives/evaltalk.html. This is a traditional listserv that goes directly to your email inbox. Subscribing to EvalTalk is a must (if only to watch the bloodbath as evaluators battle each other online). Make sure you adjust your settings so that you get a daily or weekly digest – otherwise you’ll drown in the sheer volume of messages.

Second, subscribe to mailing lists and newsletters specific to your client projects. Whenever I begin a new project, I search the client’s website and subscribe to everything I can (like their Twitter feed, email newsletter, and blog). As a consultant, I only see one slice of their work. Subscribing to all of their updates helps me get a fuller picture of their work, so I can make sure the evaluation fits their organization’s culture and needs.

Thought Leaders Discussion Series

AEA’s Thought Leaders Discussion Series is like a big message board to debate bigger-picture, theoretical issues in the field. Each series is led by a different person and has a different flavor.

Topical Interest Groups (TIGs)

Topical Interest Groups (TIGs) are known as affinity groups in other professional associations. You get to select five TIGs when you join AEA, and you can change your selection at any time. Each TIG is different–different sizes, leadership and committee structure, and different business meetings. I suggest attending business meetings for multiple TIGs at each conference. See which culture fits you best. After a few years, get more involved by running for a leadership position.

Twitter

Just getting started on Twitter? Here’s my list of 275+ evaluators and 80+ evaluation organizations who are using Twitter. Use #eval13 to tweet about that year’s AEA conference (not #AEA13 – the poor folks at the American Equine Association will get confused). Use #eval for all your regular evaluation-related content.

There's a huge online evaluation community. What are you waiting for?!

Here’s Johanna Morariu’s social network map of the #eval13 hashtag. There’s a huge online evaluation community. What are you waiting for?!

White papers and other gray literature

There are approximately 8000 evaluators in the American Evaluation Association. I estimate that maybe… 5%?… aim to publish articles in academic journals. Most of us are practitioners and consultants (not academics, theorists, or professors). White papers and other gray literature are a great way to learn about our work, our insights, and our tips. For examples, check out innonet.org/research and evaluationinnovation.org/publications.

Additional resources

What are your favorite resources? Which resources were most valuable during your first few years in the field? And, most importantly–do you have different viewpoints on any of the resources I described? Share your perspectives! I’ve presented one opinion and there are many more to add to the mix.

Getting the Resources You Need for Evaluation [Guest post by Patrick Germain]

Patrick Germain

Hi evaluators!  My name is Patrick Germain and I am the Director of Program Evaluation and Quality Assurance at Project Renewal, and the Founder and Director of Performance Management Professionals, a community of practice based in New York City on issues of Performance Measurement and Management and Evaluation.

Organizations are rife with politics, and politics is often the decisive factor in whether evaluations get used. This is the third post on how I navigate and influence organizational politics to ensure that evaluation is appropriately supported and used in decision making. Check out my earlier posts on Managing Your Evaluation Reputation and Preempting Conflict in Evaluation.

Week Three: Getting the Resources You Need for Evaluation

‘Doing more with less’ seems to be standard operating procedure in the non-profit sector these days. Need for social services continues to rise, and funding continues to be scarce – and in the rush to scrape up whatever funding there is, evaluation is often left in the dust.

So what can internal evaluators do to make sure we get the resources we need? Unfortunately, not a whole lot… But that doesn’t mean we have to throw up our hands.  It is what it is, but that isn’t how it has to be!

Even if you can’t magically double your budget, there are some things you can do to get your needs higher on the organization’s priority list:

  • Understand the priorities of the decision makers – know what they want, and demonstrate how you having additional resources will help achieve that goal.
  • Get lots of different people to ask – A brief illustration: From day one, I knew that our IT capacity was insufficient – not only for me to do my job, but also to solve a myriad of problems throughout the organization.  Every time someone highlighted a relevant problem, I explained how IT solutions might be able to fix it and asked them to help me build pressure for more IT capacity.  I cultivated the understanding that there was huge unmet potential that could be unleashed with more investment in IT.  At a certain point, it became a clearly articulated organizational need that the decision makers had to place higher on the priority list.
  • Don’t make it about yourself – make it about impact and efficiency.  Demonstrate the impact that your work will have on the clients or constituents. Show how it will help the staff and the programs.  Demonstrate that evaluation is an investment, and not just a cost.
  • Get evaluation built into future budgets – It is hard to shift around resources that are already allocated, but it’s a lot easier to get evaluation built into future funding streams.
  • Align with the strategic plan – My organization recently went through a strategic planning process, and it was my one goal in life to make sure evaluation got incorporated into it. (ok, so maybe that’s an exaggeration).  But our strategic plan now has initiatives directly relating to building evaluation capacity.  Evaluation is also integrated into measuring whether we actually achieved what we set out to do in our strategic plan.
  • Ask for it and be ready to justify it – It’s amazing how often people forget this one critical step.  If you don’t make it clear what you need and why, it is doubtful you will get it.  But remember, when you are justifying your request, frame it within the priorities of the decision makers.
  • Leverage other organizational resources – I was surprised at how many of our program directors were excited to participate in evaluative activities.  I asked for volunteers, I asked to borrow staff for a short period, I asked to borrow space and equipment, I asked for help on specific questions or challenges.  If you have done a good job at building your reputation and building allies, you might even get people to agree to it!

It is entirely possible that these ideas won’t work in your context.  Do any of these strategies apply to your position? Are there other ways you have gotten resources?

– Patrick Germain

P.S. Want to learn more? Look for me at the American Evaluation Association conference in Minneapolis, where I will be presenting on some of these issues.