Category Archives: Interviews and Focus Groups

Checklist for Focus Group Facilitators

Are you facilitating a focus group? If so, you’ve probably got a lot on your mind – greeting participants, making sure everyone feels comfortable, covering all the key questions… It’s a lot to remember!

As I was learning to facilitate focus groups, I started taking notes of all the good advice I received from experienced facilitators. I turned that advice into a checklist for myself. I  review my checklist before every focus group (to keep the advice fresh on my mind) and after every focus group (as a self-evaluation). You could also use this checklist to assess colleagues who are learning to facilitate focus groups (i.e., use the checklist as a conversation starter when you’re debriefing together after the focus group).

Here’s my checklist for focus group facilitators. Did the facilitator:

  • Introduce him or herself?
  • Explain the purpose of the focus group?
  • Explain the rules for the discussion?
  • Address issues of confidentiality?
  • Use verbal and non-verbal expressions to make participants feel comfortable during the focus group?
  • Facilitate real interaction among group members (not just a series of individual interviews)?
  • Draw everyone into the discussion?
  • Demonstrate genuine interest in the discussion?
  • If asked for their personal opinion on a topic during the group, deflect requests by participants to give an opinion?
  • Maintain a good pace during the discussion (kept things moving but don’t rush questions)?
  • Ask questions in a neutral way (not in a leading way)?
  • Link comments to previous comments or themes?
  • Address comments or behaviors that could take the discussion off course?
  • Address disruptive and disrespectful behavior?
  • Ask participants for validation if rewording or summarizing comments?
  • Courteously but firmly stop a chatty participant?
  • Demonstrate sensitivity to emotional reactions of participants?
  • Display awareness and respect for cultural issues that emerge in the discussion?
  • Re-explain or re-phrase questions as needed?
  • Sincerely thank participants for contributing?

Do you have additional tips to add to the checklist? Which tips have been most valuable to you when facilitating a focus group?

P.S. For more focus group resources, check out these posts.

Ideas for interactive focus groups

    

Hello fellow evaluators,

My teammate and I are running a focus group in a couple weeks. The group will be large (about 17 people) and will be with working professionals (mid-20s through mid-40s).

We’d like to add a few interactive elements to this focus group so that people have a variety of ways to express themselves, move around, and have some fun.

Here are some ideas for interactive focus groups so far:

  • Build a collage (read more and watch video examples here)
  • Hold small group discussions
  • Vote with stickers, i.e. showing where they stand on issues by putting a colored sticker on a large sheet of paper
  • Start with a brief survey (read more here)
  • Start with a free-writing exercise “to help participants access relevant memory. Jumpstart a rich discussion by asking them to share what they wrote with one another.” (read more here)
  • Write down how they feel on index cards (read more here)
  • Roundtable Ranking, i.e. small groups rapidly brainstorm the strengths and weaknesses of the program on a single sheet of paper and rank-order (1, 2, 3) the strengths and weaknesses (read more here)
  • The Interview Design Process, a hybrid of a focus group and a speed dating session (read more here)
  • The Values Walk, in which people physically walk to different corners of the room to indicate whether they agree or disagree with your questions (read more ideas from Public Profit here)
    • This worked well as part of a community listening project in the youth center where I used to be an internal evaluator. Groups of 10-15 adults walked either to the far right of the room (agree) or far left (disagree), or anywhere in between on the spectrum. As the facilitator asked a series of questions, you got to watch the patterns in the group as well as each person’s individual variations in responses.
    • The only downside is that it’s very, very, very hard to take notes during this activity. Everyone’s walking around the room, and the notetaker is furiously typing on a laptop, trying to capture the physical movements as well as the conversation. Perhaps an audio recorder and camera would work better? I wish I had pictures of these Values Walks from the community listening projects.
  • Build a community map (read more ideas from Public Profit here)
    • This also worked really well in the youth center where I used to be an internal evaluator. The youth collected data about tobacco retailers in their community and mapped the data using density maps and little pins and flags. The data was part of their needs assessment, and then the next step was to develop messaging for an anti-tobacco campaign.

Please share your own ideas in the comments section below. Have you tried any of these techniques yourself? Did any of these go particularly well (or poorly)?

Thanks, Ann Emery

Facilitating focus groups vs. facilitating other types of group discussions

A lovely side effect of building an organizational culture of learning is that there’s a huge demand for qualitative research like interviews and focus groups. However, with so few internal evaluators, it’s hard to keep up with the demand.

I’ve trained a handful of non-evaluators at our youth center to conduct focus groups with youth – a great example of what capacity building looks like on a day-to-day basis. This is actually really, really easy because our youth workers are already skilled facilitators. They’re great public speakers, they pay attention to group dynamics, and they know how to bring the conversation back on track.

So instead of chatting about focus group basics like making sure you give everyone a chance to participate, my training emphasizes how facilitating a focus group is different from facilitating other types of group discussions (like afterschool workshops for teenagers or GED classes) that our staff members are already great at leading.

How is facilitating a focus group just like facilitating other types of group discussions? In both cases, facilitators should:

  • Pay attention to group dynamics. Keep the more talkative people from dominating the conversation. Encourage the more quiet individuals to share their opinions. Ask, “Is there anyone who hasn’t said anything yet who has something to contribute?” At the same time, recognize that some people won’t want to talk and you can’t force these people to participate. Don’t judge the people who don’t talk. Remember that everyone has different experiences. Encourage people with different opinions to speak up.
  • Keep the conversation moving along at a good pace. Keep the conversation on topic. Keep time. Be aware of the agenda.
  • Create a welcoming space where everyone feels safe to contribute. Assure participants that you’re looking for their opinions and that everyone’s opinion is important.
  • Establish ground rules and enforce those rules throughout the discussion. Address oppressive comments, like if someone makes a sexist comment. Address inappropriate behavior. Then, “lean in” to discomfort (when participants say something controversial or offensive, bring them back into the group again).
  • Ask people to elaborate by asking things like, “What do you mean by that?” or “Tell me more.”
  • Use whiteboards or large sheets of paper if you want to take notes of the big ideas that were shared.

But, when you’re facilitating a focus group:

  • There’s no “right” answer. All opinions and perspectives are correct and welcome. You’re not teaching factual information about a certain topic, you’re just discussing a topic and listening to their feedback.
  • The process is more important than the result. Your goal is to keep the conversation moving along on topic, but it doesn’t matter whether the participants agree or disagree with the questions you’re asking. For example, if you’re talking about whether the youth enjoyed a program, it doesn’t matter whether they loved or hated the program. Your job is to get them to express themselves and share detailed examples.
  • Sometimes it’s easier when you don’t know anything about the topic. I personally find that it’s always easier for me to conduct focus groups when I don’t know anything about the topic.
  • You may or may not synthesize the ideas at the end of the discussion. When there are a few different opinions on the table, it’s helpful for the participants (and for you! and for your notetaker!) to rephrase those opinions so that you can bring the conversation back together and make sure you understood everything correctly.

And when you’re facilitating another type of group discussion:

  • There is usually a “right” answer. Opinions and perspectives are valued, but when you’re teaching the group of participants about a certain content area then you need them to leave the group with factual information.
  • The end result is just as important as the process. You want to ask questions and move the conversation along so that participants get to the “right” answer at the end of the discussion. For example, if you’re talking with youth about the dangers of smoking cigarettes, then they need to leave the discussion with the understanding that cigarettes are bad for them.
  • You need to know a lot about the topic you’re discussing.
  • You need to synthesize and summarize the ideas at the end of the discussion.

These are the key similarities and differences that our tutors, mental health counselors, and youth workers helped me keep track of. What else should we add to our list?

Enjoy, Ann Emery

P.S. We developed these lists for group discussions with youth, but most characteristics would probably apply to group discussions with adults too.

P.P.S. For additional information, please check out Maria Gajewski’s post on the Changing River Consulting blog, What does a facilitator do?

My bank of youth focus group questions

I’m teaching my teammate about the ins and outs of planning, conducting, and sharing results from focus groups with youth. Today’s lesson was about 1) collaborating with program staff and/or funders to figure out what type of information they’re most interested in and 2) turning those interests into focus group questions (i.e. operationalizing). Here’s my bank of questions that I often use during focus groups with youth.

Start with an icebreaker. Make sure it’s developmentally appropriate. You’re not going to ask teenagers to share something that makes them feel like children.

  • Example from a college prep program: You might ask about their educational background, like whether they’ve taken any college courses before.
  • Example from a prenatal education program for teen parents: What are you most looking forward to about being a parent?
  • How long have you been in the XYZ program?

To learn more about this program’s role within the larger organization:

  • Have you participated in other afterschool or summer programs here? If so, which ones, and when?

To learn more whether the outreach and recruitment strategies were effective:

  • How did you find out about the program? (For example, through a teacher, another student, or another staff person?)
  • Complete this sentence: “The biggest challenge I’ve had this year was…….” (If the program is supposed to serve at-risk, gang-involved youth, and the biggest challenge this year was that they didn’t like their teacher, then they either don’t trust you enough to be totally honest with you (duh, you’re a stranger…) or the program didn’t recruit the at-risk, gang-involved youth it was originally aiming to serve.)

To learn more about their expectations:

  • What were your expectations about the program? Do you feel that they were met? In what ways?
  • What were the aspects of the classes that you didn’t anticipate? Did anything surprise you?
  • Example from a college prep program: Before the program started, what did you expect a college course to be like? How has your opinion of college courses changed over the past year?
  • What had you heard about the program? (For example, that you’d get to play soccer, do art, or participate a specific curriculum?)
  • Why did you want to sign up for this program?

Basic consumer satisfaction questions (this information will be very useful for instructors, so don’t skip it!):

  • What was your favorite thing about the program?
  • What’s your best memory from the program? (For example, was there a particular field trip, activity, etc. that you really liked? If the participants pause a long time before answering, make sure to record that in the notes…)
  • What did you think about the classes?
  • What was your favorite class and why?
  • What class did you like the least and why?
  • Some of you have taken xyz classes or programs before. Were there any topics covered in the classes that you’d already learned someplace else?
  • In which class did you feel you learned the most?
  • College prep program example: What did you think about the assignments and homework?

Consumer satisfaction with specific staff members:

  • What did you think about each of the instructors?
  • How well did the instructors explain information to you?
  • Were the instructors easy to understand? (This is an important consideration in a multi-cultural, multi-lingual organization like ours.)
  • Were the instructors creative? (Youth despise lectures and love interactive activities.)
  • Were the instructors ready to teach when you arrived?
  • How can the instructors improve the way they teach?

To learn about the extent to which Positive Youth Development principles are thriving (or not) in this program:

  • How well could you generally relate or connect to the instructor?
  • How well did the instructor relate or connect to you?
  • How do you think the instructor perceived you?
  • What kinds of support did you get from the instructor? (For example: snacks, organizing study groups, college counseling advice, spending extra time with you, helping you work through a personal problem, etc. You want to know whether the support provided was adequate and whether they provided the “right” type of support. In other words, a GED instructor shouldn’t be providing mental health counseling; the GED instructor should listen to the youth’s initial problems and refer the youth to a professional.)
  • Were the instructors available when you needed help? (This is an especially helpful question in human services organizations where most clients receive case management. The participants need to feel that all staff in the organization are available to help.)
  • Complete this sentence: “An accomplishment I’m proud of is……..”
  • HIV prevention program example: Your group planned a few health fairs this year. Tell me about your involvement in these health fairs. (To see how much ownership they felt while planning the events.)

To find out whether participants might apply the information they learned to their real life:

  • Parenting program example: Did you change the way you interact with your child as a result of these classes?
  • Domestic violence prevention program example: Has your communication with your partner, parents, and other people changed as a result of any of these classes?
  • What do you see as potential barriers or limitations in applying what you learned? (This seems silly, but don’t skip it! A participating in our prenatal program told me she couldn’t use the smoke detector that she received free during our workshops because she didn’t have any batteries. Now our instructors give out batteries with every smoke detector. Small barrier, easy fix, win-win situation for everyone.)
  • College prep program example: What are your future educational plans? How many students are planning to enroll in additional college courses, what type, and where?
  • College prep program example: How well did this program prepare you for future college courses?
  • Peer health education program example: Since the program ended, do you plan to continue working/volunteering as a peer health educator?

To get feedback about program logistics: (This should not be the central purpose of your focus group. You can learn so much more than whether the participants enjoyed the snacks…)

  • Were the meeting times convenient? Why or why not? What times would you recommend in the future?
  • College prep program example: Would you attend structured study groups if the instructor planned them? How helpful would study groups be?
  • How did you feel about the class size? Would you have more students? Fewer students? Why?
  • As you know, Jane Doe is leaving for grad school and won’t be an instructor here next year. What would an ideal instructor be like? What’s on your wish list?

To learn about potential reasons for attrition:

  • How do you think we could keep more teens in the program for the entire year?
  • What were your reasons for staying in this program all year?
  • What kept you coming back?

To plan recruitment strategies for next year:

  • Where do you suggest we find other people who would benefit from or be interested in this program?

If the program didn’t go so well….

  • What could help you be even more successful in this program? (For example: metro fare cards, child care, dinner, etc.)
  • College prep program example: Describe an ideal college student. What types of skills, personality traits, previous experience, etc. does someone need before starting these classes?

To learn about the potential influence of the program on family members (assuming this is one of the goals of the program):

  • Did your parents come to any events at our youth center this year?
  • Are your brothers, sisters, or cousins involved with any of the other programs at our youth center?

To learn about overall influence of the program:

  • What were the most important skills you learned through this program? (For example, time management, study skills, setting appointments, more self-sufficient, more independent, etc. Get specific examples. This is extremely valuable for the instructors.)
  • College prep program example: For those of you who’ve taken other college courses – How would you compare this experience to other college courses? In what ways was it similar, different, etc.?
  • Would you recommend this program to a friend or someone else?  (Get a show of hands or find some sort of quantitative way to measure.)
  • On a scale from A to F, how would you grade this program?
  • Complete this sentence: “If I didn’t come to XYZ program after school, I would probably be doing…” (If it’s a gang prevention program, you’re guessing they’ll say something like, “I’d be hanging out with the wrong crowd.” One “gang-involved” girl told me she was in several afterschool programs, like volunteering, tutoring, dance, and painting. If she didn’t come to our gang prevention program, she would’ve just been at another great program. Although this isn’t what I was expecting to hear, and it’s not what the program staff wanted to hear, it’s very useful information to have.)
  • Complete this sentence: “Before I started the XYZ program, I would describe my personality as……..”
  • Complete this sentence: “After I finished the XYZ program, I would describe my personality as……..”
  • How would you describe this program to a friend? To your mom? To your sister or brother?
  • What have you already told your friends about this program?
  • What did you learn about yourself by participating in this program?

And always end with this:

  • “Do you have many other thoughts or comments you want to share today?” (There are always more comments, so be patient and way for them to speak. If no one speaks, stay silent for 5 seconds and look around the room, making eye contact with each participant until someone speaks up, then thank them for sharing. Then, do it again – be quiet for 5 seconds and look around the room until someone speaks, and thank them for sharing. Do this 5 or 6 times until people are really, truly out of feedback.)

Additional considerations:

  • I do not ask the questions in this order or in these exact words. As with all qualitative research, I have a few questions in mind before I start, but mostly I just relax and go with the flow of the conversation.
  • Try to make the discussion interactive. All focus group participants — children, teens, and adults — will get bored sitting around a table for an hour. Get some ideas here.

Enjoy! And please share your own favorite focus group questions below.

Ann Emery

 

Understanding Mexican Culture to Improve the Interview Experience with Mexicans

I watched a great American Evaluation Association Coffee Break webinar today by Efrain Gutierrez. The webinar was called “Understanding Mexican Culture to Improve the Interview Experience with Mexicans.” Efrain shared some useful tips for interviewing Mexicans. Here are some take-aways from his webinar:

  • Make sure you give Mexican women a chance to speak up during the interview since they’re not always used to speaking up in the machismo Mexican culture.
  • Leave time for small talk, especially at the beginning of the interview.
  • Don’t be afraid to go off-topic for a few minutes.
  • Remember that Mexicans have a different perception of government services than Americans. Check out the slide below.
  • Mexico is an extremely diverse country, so people from the city will respond differently than people from the country.
  • Remember that first-generation immigrants will respond differently than second-generation or third-generation immigrants.
Understanding Mexican Culture - webinar by Efrain Gutierrez

Understanding Mexican Culture - webinar by Efrain Gutierrez