Methods for evaluating and monitoring learning projects

On this page, we look at and compare three methods for evaluating and monitoring learning projects: surveys, individual interviews and group interviews.

Surveys and questionnaires

Surveys and questionnaires can be carried out in face to face training sessions, online, out in the local area or in special sessions set up for the purpose.

They should be pre-tested to ensure that young people will understand the questions. Questions can be structured (ie including solely of closed questions) or semi-structured (ie including some closed and some open questions – suitable for those who are good at writing).Closed statements are easier and quicker to answer and analyse. Responses can be Yes/No, multiple choice or on a three or five point sliding scale (eg from strongly agree to strongly disagree)

Advantages

  • they are generally confidential, private and anonymous
  • they are economical, allowing you to ask many people
  • they are easy to analyse
  • they might be popular with young people as they follow a familiar format
  • they might be good for young people who are shy or dislike writing
  • they can be devised in participatory ways, with young people

Disadvantages

  • They only deal with a narrow range of issues
  • Reasons for answers are not given so it can be hard to analyse
  • They rely on reading skills which may be a barrier for some participants
  • They are difficult to design well
  • They can be manipulated (eg by asking leading questions)
  • Getting people to complete them can be difficult

Individual interviews

Individual interviews are a popular way of gathering responses and perspectives on learning courses. They can be conducted in situ, at a neutral venue or even on the phone. Interviews allow for individual questions to follow areas of interest. Responses may be written down or recorded using audio or video. Consider how you arrange seating carefully, think about interviewing in friendship pairs and make sure you think carefully about how you address the young people when conducting interviews.

Advantages

  • They often provide thoughtful responses
  • Interviewers can follow interesting areas of concern to the young people
  • Participants often enjoy them
  • They can be more personal and private than group interviews

Disadvantages

  • Some young people are not used to talking to an adult on a one to one basis and may find it difficult
  • It is time consuming and you will only be able to talk to a few participants
  • They require considerable skills to conduct them well
  • Sometimes young people will respond with what they think adults want them to say
  • It may not be possible to maintain confidentiality

Group Interviews and Focus groups

Group interviews and focus groups can be helpful as they may counter some of the power issues that can arise in one to one situations with an adult posing questions to a child. Focus groups can be useful to draw on the interactions within a group, and to draw people out within a social context that is more familiar to them. It’s sometimes possible to pick up on authentic language and voice in a way that is often stifled in a one-to-one interview.

One thing you need to be aware of in carrying out group interviews is that some people in a group will often talk more than others. It can be a real challenge to get the quieter members of a group to talk, or to prevent one or two members’ voices dominating – so this is a skill that people who carry out group interviews need to develop. One thing you can do is to address questions specifically to the quieter members, often in combination with adopting encouraging tone of voice and body language to make them feel more at ease with talking with you (and the others in the group). It’s often worth thinking about the mix of people before you start. Are these people that are likely to be able to talk together freely? Is there likely to be any conflict of opinion; is this a problem if so? (It might not be.)

Advantages

  • they are a more natural setting for many people – giving authentic language and opinions
  • they are easier to coordinate than multiple individual interviews – also requiring less preparation
  • like an interview, but unlike a survey, the interviewer can interact directly with respondents, which allows for clarification and follow-up questions. Similarly, non-verbal responses can affirm or contradict what people have said (though this presents a challenge for radio, if applicable!)
  • they are very flexible, and can be used with wide range of topics, individuals, and settings

Disadvantages

  • you have less control over the interview; less able to control what information will be produced
  • they are not great for being able to tell about what an individual thinks; is more suitable for finding out what the group thinks
  • they need a carefully trained interviewer who is knowledgeable about group dynamics
  • the interview may be biased by there being a very dominant or opinionated member; more reserved members may be hesitant to talk