Tag Archives: newbie evaluators

The Conference is Over, Now What? Professional Development for Novice Evaluators

At the American Evaluation Association’s annual conference in October 2013, I led a roundtable titled “The Conference is Over, Now What? Professional Development for Novice Evaluators.” We discussed ways that novices can deepen their knowledge, build their skills, socialize with other evaluators, and get involved in leadership positions. I compiled the notes here so more people can benefit from these resources.

Here are the best resources for novice evaluators:

aea365

aea365This is the American Evaluation Association’s daily blog located at aea365.org. You can read about everything from item response theory to slide design. Confession: I rarely read an entire post. Instead, I’m skimming the posts just to see the title, author, author’s organization, and the main gist of the content. This is a great way to stay up-to-date on the biggest trends in the field.

You should seriously write for aea365, probably 2-3 times a year, even if you’re new to the field. Just make sure you follow the contribution guidelines.

Affiliates and other organizations

AEA is the national-level mothership and there are more than 20 local and regional affiliates. You can find a full listing of affiliates here: http://www.eval.org/p/cm/ld/fid=12. Every affiliate is different. For example, the Washington Evaluators hold monthly brown bags, quarterly happy hours, and an annual holiday party. The Eastern Evaluation Research Society holds an annual 3-day conference. Other affiliates hold virtual book clubs, maintain blogs, or simply hold member meetings via teleconference.

You should join your affiliate. Seriously. The mailing lists are little nuggets of gold and worth every penny of that $25/year membership. The Washington Evaluators, for example, send job announcements almost every day, so you’ll always know which organizations are hiring and expanding. Don’t forget to attend the affiliate events too. (Sometimes people just pay dues but skip all the events, and then they don’t know why they’re not meeting anyone? This confuses me.) After a year, start planning small events yourself, like a brown bag. Then, join the Board.

Here are some additional reasons to join affiliates and ideas for getting involved:

data_community_dcThere are tons of additional evaluation groups. For example, the Environmental Evaluators Network, led by Matt Keene, holds forums for evaluators interested in environmental issues. If you’re in Washington, DC, the Aspen Institute holds quarterly breakfast panels focused on advocacy evaluation. At Innovation Network, we hold Ask an Evaluator sessions for nonprofit leaders. Tony Fujs and I also attend Data Science DCData Visualization DC, and Data Community DC monthly meetups. No matter your city, there are probably lots of events that fit your interests.

Blogs

First, check out evalcentral.com, run by Chris Lysy. Chris pulls in feeds from 60+ evaluation blogs so you’ll get exposed to a diverse set of perspectives. Chris even developed a daily email digest, so you can subscribe once to all 60+ blogs rather than monitoring your subscriptions to all the individual blogs. I suggest setting EvalCentral as one of your homepage tabs (along with your other must-haves like Gmail and Pandora) so it’s there every time you log into your computer. And again, I rarely read an entire blog post but I skim everything for the title, author, and main gist of what they’re talking about.

Second, check out AEA’s listing of evaluators and evaluation organizations who blog: http://www.eval.org/p/cm/ld/fid=71

I started blogging after watching Chris Lysy’s Ignite presentation at the 2011 AEA conference. Here’s Chris’ Ignite, which outlines just a few of the infinite reasons why evaluators should blog:

Coffee Break webinars

Coffee Break webinars are just 20 minutes long, so they’re a perfect way to squeeze in some quick professional development in the middle of a busy work day. The best part? They’re free for AEA members. I like to sign up for topics that I know nothing about. After 20 minutes, I’m not an expert, but at least I’ve got a basic understanding of that flavor of evaluation.

Conferences

Evaluation conferences include:

Do you know of additional evaluation conferences? Please link to them in the comments section below.

I also like to attend non-evaluation conferences to hear how non-evaluators are describing our work (they have completely different lingo and tend to value qualitative data way more than evaluators do).

eStudies

An eStudy is a 3- to 6-hour webinar run by AEA. eStudies are like mini grad school courses because they go in-depth on a particular topic (as opposed to 20-minute Coffee Break webinars, which just provide an overview of a topic). eStudies are broken into 90-minute chunks and there’s typically a homework assignment between each segment to help you practice your new skills.

For example, I participated in an eStudy about nonparametric statistics in which the instructor covered about 20 different nonparametric statistics, when to use each one, and how to perform the calculations in SPSS. We even got to keep her slides, which were full of step-by-step SPSS screenshots. Almost two years later, I still pull out my eStudy notes whenever I need to use some nonparametric statistics.

Journals

AEA offers two journals, the American Journal of Evaluation and New Directions for Evaluation. Both of these journals are included with your AEA membership. What a steal!

LinkedIn

These days, I can’t imagine an employer not doing a full internet search on new applicants. Make sure your LinkedIn profile has, at the bare minimum, a professional photo, your full work history (including dates), and your education history. You can also use LinkedIn to build your online portfolio (e.g., embedded slideshows from recent conference presentations, links to publications and projects, and your list of certifications).

Want to connect with other evaluators? Some awesome evaluation groups on LinkedIn include:

Do you know of additional evaluation groups on LinkedIn? Share your suggestion in the comments below. Thanks!

Listservs, mailing lists, and newsletters

First, check out EvalTalk: https://listserv.ua.edu/archives/evaltalk.html. This is a traditional listserv that goes directly to your email inbox. Subscribing to EvalTalk is a must (if only to watch the bloodbath as evaluators battle each other online). Make sure you adjust your settings so that you get a daily or weekly digest – otherwise you’ll drown in the sheer volume of messages.

Second, subscribe to mailing lists and newsletters specific to your client projects. Whenever I begin a new project, I search the client’s website and subscribe to everything I can (like their Twitter feed, email newsletter, and blog). As a consultant, I only see one slice of their work. Subscribing to all of their updates helps me get a fuller picture of their work, so I can make sure the evaluation fits their organization’s culture and needs.

Thought Leaders Discussion Series

AEA’s Thought Leaders Discussion Series is like a big message board to debate bigger-picture, theoretical issues in the field. Each series is led by a different person and has a different flavor.

Topical Interest Groups (TIGs)

Topical Interest Groups (TIGs) are known as affinity groups in other professional associations. You get to select five TIGs when you join AEA, and you can change your selection at any time. Each TIG is different–different sizes, leadership and committee structure, and different business meetings. I suggest attending business meetings for multiple TIGs at each conference. See which culture fits you best. After a few years, get more involved by running for a leadership position.

Twitter

Just getting started on Twitter? Here’s my list of 275+ evaluators and 80+ evaluation organizations who are using Twitter. Use #eval13 to tweet about that year’s AEA conference (not #AEA13 – the poor folks at the American Equine Association will get confused). Use #eval for all your regular evaluation-related content.

There's a huge online evaluation community. What are you waiting for?!

Here’s Johanna Morariu’s social network map of the #eval13 hashtag. There’s a huge online evaluation community. What are you waiting for?!

White papers and other gray literature

There are approximately 8000 evaluators in the American Evaluation Association. I estimate that maybe… 5%?… aim to publish articles in academic journals. Most of us are practitioners and consultants (not academics, theorists, or professors). White papers and other gray literature are a great way to learn about our work, our insights, and our tips. For examples, check out innonet.org/research and evaluationinnovation.org/publications.

Additional resources

What are your favorite resources? Which resources were most valuable during your first few years in the field? And, most importantly–do you have different viewpoints on any of the resources I described? Share your perspectives! I’ve presented one opinion and there are many more to add to the mix.

The Evaluation Mentor: A Wish List [Guest post by Katie Aasland]

Greetings, I’m Katie Aasland, and I’ve worked in the evaluation field for less than a year. I have composed a list of the traits I find most helpful as a fledgling evaluator in the hopes of fostering more of those behaviors in readers who currently have mentees.

  • Be Supportive. As newbies in the field, we will ask you lots and lots of questions. We are the proverbial clay and need molding. I may have learned about evaluation in course work or have some prior experience, but I don’t know your organization yet. It is difficult to find mistakes in reporting or calculations when I do not know what the outcomes typically look like. No one wants to report incorrect information, so there may be a period where I want to double-check everything with you before sending it out.
  • Have Patience. My questions will lessen as I am more comfortable with the job.  I just want to make sure I’m doing things right. Jobs and the life of programs may depend on our work – it’s a lot of pressure to get used to!
  • Encourage us to Slow Down. Being told to slow down is helpful. In school and other jobs, I tended to finish things quickly and was rewarded for it. In order to do quality evaluation, one needs to take the time to focus and really think about what data are showing.
  • Be Organized. When the processes, communications, computer files, and reports are streamlined, things go smoothly and I can help you conduct a better evaluation.
  • Provide Structure. I have a weekly and monthly schedule of when things need to be accomplished, so when I finish one task, I know where to go to find the next task I can work on.  Also, having routine things to work on provides me with insight into the organization and familiarizes me with our information.
  • Have a Sense of Humor. Evaluation can be a stressful field, especially when money or jobs are on the line. It is great to laugh, even for a second, and a sense of camaraderie can buffer against burnout.

— Katie Aasland

Perspectives from New Evaluators

It’s back-to-school time again! I invite you to say hello to recent grads and newbie evaluators.

It’s that time of year again. Stores are full of back-to-school clothes, backpacks, and school supplies. They’ll be selling Christmas decorations any day now!

Newbie evaluators and recent graduates experience a back-to-school nostalgia around this time of year. We’re reflecting on what we’ve accomplished since graduating and wondering what the evaluation field holds for us in the future.

But what does it really mean to be a new evaluator? What do all these new, newbie, and novice evaluators have in common? What about the undergraduate students, graduate students, and recent grads?

I’ve been working in evaluation for 4 years but I still consider myself to be a new evaluator. There’s so much to learn! Every client, client team, organizational culture, evaluation team, programmatic content area, logic model, evaluation plan, dataset, and needs assessment is new. Even when working with repeat clients or conducting multi-year evaluations, the contextual factors affecting the program and its evaluation will evolve. (Does the client ever want and need the same information in Year 5 that they wanted and needed in Year 1? I doubt it. Does Decisionmaker A want and need results shared in the same way as Decisionmaker B? No way.)

At what point do we transform from new evaluators into seasoned evaluators?

All this month, you’ll hear from new evaluators and recent grads about their successes, challenges, and questions as they’re transitioning into the field. I invite you to read their perspectives and say hello.

— Ann Emery
P.S. Additional insights from new evaluators are located here.