Evaluation, accreditation, case studies and sector needs

15. Evaluation and monitoring

59% of those who answered the question, “do you currently evaluate training projects with young people” said that they did, 11% said they only did so if funders demanded it, and 30% stated that they didn’t do any evaluation.  Of those who did evaluate projects, a variety of evaluation strategies were used, as detailed in table D, with practitioners using on average 5 different techniques.

Table D: What kinds of evaluation strategies have you adopted with groups of young people (tick all that apply)?

Evaluation strategy

%

Recording attendance and demographic data

83%

Tutor assessment

72%

Distance travelled evaluation techniques e.g. Outcome Star

33%

Surveys/ questionnaires

67%

Focus groups

17%

Learning journals

44%

Blogs

17%

Shared social network

28%

Audio/ video diaries

33%

Interviews with participants/ stakeholders

72%

When we asked practitioners in our phone survey about evaluation they often said that they tend to do evaluation ‘informally’ by ‘asking young people what they think’ during and at the end of training courses.  Some stated that they evaluate projects by listening to what young people say rather than adopting more formal methods.  These informal approaches to evaluation may not have been seen as an ‘evaluation’ strategy by those responding to the online survey.

Some stations described more formal evaluation strategies such as written end of course evaluations (by learner or tutor), or allowing a half day at the end of a course for focus groups and paper based evaluation surveys to be completed.  These stations also considered following participants after they had finished courses to monitor their progression.

A minority of stations (two that we spoke to) saw evaluation as part of their course delivery – developing an individual learning plan with course participants, measuring attendance, and holding focus groups and interviews.  Others discussed more creative approaches to evaluation such as using audio diaries and graffiti boards.

Challenges that respondents discussed in relation to developing evaluation strategies included:

  • finding it difficult to measure change
  • being short-staffed in terms of paid people
  • not being aware of different evaluation techniques
  • youth workers at their organisations not being trained to conduct evaluations and finding it onerous
  • finding time to collect testimonials and other evidence in the ‘busy’ atmosphere of radio projects

Our survey data suggests that practitioners would benefit from a set of centralised resources looking at evaluation strategies and methods, perhaps also including templates and exemplar databases/ analytical tools.

 

16. Accreditation

Only 39% of the stations that deliver training to young people said that they delivered accredited training.  Most of these (64%) deliver radio production qualifications.  Others deliver:

Key/ core/ transferable skills or similar qualifications 27%
Youth achievement awards 36%
Community work/ community action awards 36%
In house schemes/ self-certification 36%
Employability qualifications 9%
Arts or creativity-based awards 18%
Other (eg HND qualifications, creative/media diploma) 9%

The most common award mentioned in phone interviews was the NCFE Level 1/Level 2 in radio production which was delivered on site at several stations.  In some cases, a grant was awarded for the radio station to deliver the course for a limited amount of time.  For instance at Somer Valley, they taught a Level 2 NCFE course in radio production to 17 people with a youth service grant of £35k.

Another commonly mentioned award was an ASDAN-certified  ‘Skill Me Up’ award, sometimes delivered in partnership with a local youth service.  This qualification provides trainees with recognition of what they have achieved in respect to their community and voluntary work at the station.  Some stations had delivered the Arts Council ‘Arts Award’, which encourages young people to take part in arts activities and experience artists and their work.

Stations provided a variety of reasons for not delivering accredited courses, including lack of capacity – seeing the accreditation process as complex, hard to grasp and time-consuming, and impossible for them to commit the required resources to in the current financial climate.  Other stations said that colleges draw down all of the funding to deliver accredited courses and it is often difficult for community organisations to compete.

Several stations told us that they do not deliver official accredited training programmes but have devised a process of in-house certification in order to recognise the input of volunteers at their station.

Many stations told us that they were very interested in delivering accredited courses and would like to know more about how to go about doing so.  Sunny Govan told us that there is a demand for training, and would like to see future work linked with Individual Learning Accounts[1].  They are considering working with a local university (Caledonian) in order to use their resources as a backdrop for this.

 

17. Young people and social media

85% of stations that work with young people said that they made use of social networking sites.  Of these, 96% used Facebook, 87% Twitter, and 13% of respondents said that they had their own bespoke networking space.  Organisations tended to use social networking sites for promotional purposes – places where young people could promote their own shows and the organisation (91%) and communicate with audiences (86%).  77% of respondents said they used social networking sites for young people to communicate and collaborate with each other and with staff members.  Only 3 organisations said that they didn’t use social networking sites with young people – reasons given were fears around child protection issues, lack of staff expertise, and having tried it but found it unsuccessful.

When we spoke to practitioners, many of them said that young people tended to enjoy using a ‘social media style’ of broadcasting where they drew on a range of social media channels to publicise (and broadcast) their shows and audio.  Several practitioners said that they did not need to encourage young people to consider using social media in this way.  In many cases, it was young people who had helped the organisation move forward with a social media strategy for the content and publicity of the organisation.

Practitioners did identify some challenges with using social networking with young people.  For instance, there is a need to be clear about appropriateness (without being too dictatorial).  In addition, communication and participation can be an issue on a closed site as young people may feel that having to input a password makes it inaccessible and does not fit with their everyday digital practices.  Practitioners found it more successful when they used an existing social media channel such as Facebook.

  

18. Case studies

When we spoke to practitioners many of them spontaneously started telling us about individuals who had benefitted from community radio training.  We outline some of these stories in this section of the report.  Some of these stories are powerful and highlight the need for the sector to move beyond anecdotal accounts and to develop stronger evidence concerning outcomes.

 

18.1 Young people with special needs: Siren FM

One young man with special needs was on day release from his school over the period of a year. He has now been working as a volunteer at the station for 3½ years and recently interviewed Mark Thompson from the BBC.

Siren also work with a 23 year old autistic man who has a passion for jazz, who comes and does a radio show every week with live phone interviews.  He interviews famous jazz musicians and is not afraid to just call them up.  The volunteer manager at the station picks him up from the bus station each week – his care is signed over to Siren FM for an afternoon a week.

 

18.2 Takeover Week: SoundArt Radio

During a designated ‘takeover’ week, one young man who came in didn’t appear to care about his appearance and found it difficult to make eye contact.  He bumped into an old friend and after a few days they came to staff saying that they’d decided together to overcome their fears.  They wanted to produce and present a metal show and wanted to interview a band.  By the end of the week he was smartly dressed, making eye contact, and engaging with others in the station.  He said it was ‘the best thing I’ve ever done.’

 

18.3 Young people in charge: Generate Radio

Two young people set up this station and now run and manage it.  Before getting involved in community radio they were not engaged at school, or in training.  After completing a community radio course at their local youth centre, they went and studied at college before setting up their own organisation.

 

18.4 Underachieving at school: Somer Valley FM

The station worked with a young man who had been written off by school at 14 years old.  He was under-achieving and had developed few communication skills, either verbal, written or oral.  In addition, he caused trouble at school.  He became engaged in radio production, and in particular the technical side of radio, and now has developed a career in radio.

 

18.5 A ‘safe’ space for expression: Drystone Radio

One young man who makes a programme for this station finds it very difficult to express himself outside of the radio station.  However, he produces fascinating programmes in the radio station and has found therapeutic expression in a programme.  His programmes have a big youth following on Facebook.

 

18.6 Representation and recognition: Sunny Govan

A Muslim homosexual young person went on air as part of their “Smashing Dishes” series. This was a significant step for him, given that he felt persecuted about his sexuality, and had been told by his family that he would be safer if he “forgot about his sexuality”.  Being able to tell his story on air shows the confidence he gained to come out about himself, not only to his family, but to the wider community.

 

18.7 Engaging NEETs: ALL FM

One young man was not attending school or college and spent a lot of his time smoking marijuana.  He had little self-esteem and his life was in a mess.  After involvement and programme-making at ALL FM, he became a youth engagement worker and then got a breakfast show for 6 weeks at BBC1xtra.  This is particularly remarkable because at the start of training at ALL FM, he’d have said that the time the show started was his bedtime.

 

18.8 Engaging vulnerable young people: Frome FM

A specialist college for people with learning difficulties emailed the station and asked if a young man could come in with a support worker over a period of a year.  He was a vulnerable young man who lacked confidence and belief in himself.  He loved being at the station and ended up running his own show, which he loved.  He has since been offered a full-time job at the local council.

 

19. Sector needs analysis

19.1 Practitioner needs

As part of our phone survey we asked practitioners what kind of support they felt that they needed in order to develop their own practice in youth radio training.  As we spoke to a wide range of organisations, we received a range of responses, but some similarities emerged.

Practitioners felt that they would benefit from a way of communicating with each other in order to successfully share skills and advice on the delivery of training and working with young people.

Several organisations said that they felt the value of a bank of centralised resources would be immense, as they often feel very isolated from each other.  Where resources are tight, few practitioners can spare time to develop their own resources and expertise, or find time to go out and network with others.

Practitioners said they would benefit from:

  • A sharable resource of training materials
  • Best practice guides in different areas of working with young people e.g. how to set up a course, training mentors
  • Details of nationally recognised courses and accreditation routes
  • Advice on mapping skills-learning onto radio production courses
  • Advice on policies in relation to young people e.g. safeguarding policies, legality for working with people under adult age, employment regulations around volunteers, health and safety for training, codes of conduct for young volunteers
  • Advice on funding/ monetising their practice
  • Advice on partnership building e.g. example contracts with partner organisations re expectations of each other
  • Practical tools e.g. how to do CRB checks

After speaking to practitioners we also felt they would benefit from:

  • Guidance on evaluation and monitoring – thinking beyond anecdotal case studies
  • A change in their approach to funding – from a grant-led approach to a service-led approach
  • A space to discuss their professional training practices in order to get better at talking coherently and to different audiences about what they do

 

19.2 For young broadcasters

Practitioners also felt that young broadcasters could benefit from a radio network of their own where young people could help and support other young people.  Further work is needed here with young people themselves to assess how important this is to them and what kind of space they would find useful and engaging.  However practitioners suggested that this resource might include:

  • Information about progression routes
  • Easy to understand starter packs/ technical guides for new young broadcasters
  • A young broadcasters ‘radio training academy’ that provided courses and materials that young people could dip in and out of
  • Ways of encouraging international radio relationships as a standard part of training programmes for young people


[1]  Individual Learning Accounts are only currently used in Scotland where they provide a small amount of means-tested funding (up to £200) for training individuals aged 16 or over

Add Comment Register



Leave a reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>