Skip to main content

Researcher resources

How to evaluate your activity

Introduction

Evaluation can provide valuable insights into how to both improve your project and also to assess its impact. Throughout the SUPI project a range of evaluation tools were used to collect both qualitative and quantitative data.

This guide introduces some of the methods used to access the experiences of key stakeholders involved in the project including the school partner, pupils and early career researchers.

There are wide range of evaluation tools and methods to help you reflect on your project and collect evidence of its impact. Below is by no means an exhaustive list, but it will give you a flavour of the diversity available.

  • Voting
  • Mind maps
  • Graffiti walls
  • Rich pictures & photos
  • In and out thinking
  • Questionnaires
  • Interviews
  • Focus groups
  • Postcard home
  • Concept cartoons
  • Participant observation
  • Participant-led plenary
  • Film
  • Reflective log
  • Sentence starters

Working with the Catalyst project we have produced a document that introduces each of these tools, detailing how they can be used and some of the advantages and disadvantages they bring.‌

The rest of this guide takes you through how we applied some of these methods on the EPEE project to assess the experiences of school partners, early career researchers and pupils.

The experiences of school partners were accessed using sentence starters which they complete and return:

Sentence starters used to access school partner experiences

I learnt…

I liked the way the researcher….

When next teaching this topic I will try…

The session could have been improved by…

I learnt…

I liked the way the researcher….

When next teaching this topic I will try…

The session could have been improved by…


We also used data from a range of informal conversations and reflections through email.

The experiences of pupils participating in EPEE were collected in three major ways:

  • Paper surveys
  • Cartoons
  • An evaluation task

The paper survey’s given out after an event. The survey was based on ESRC Festival of Social Science evaluation form. The ESRC questionnaire provided evidence of the ECR’s skills in communicating research to young people.

Secondly we used cartoons to capture pupil voice on what they felt they had learnt:

In future sessions we would have liked to give more time to capturing pupil reflections and focus more on finding out the impact of the event on pupils’ attitudes towards research and on their personal aspirations towards university education.

The evaluation task was focused on asking pupils to answer questions on what they had learned from the activities, we used post-its and encouraged brief responses. The questions asked were:

  • Please tell us (on green post-its) how has Silvia’s visit to Tavistock together with your visit to the university changed: 

  • what you think or know about engineering research 

  • your personal aspirations, about your next steps after A levels, and/or about your career possibilities? 

 

  • Please tell us (on orange post-its) what you have learnt about: 

  • ALM 3D printing  

  • how someone becomes an engineering researcher 

  • the work of an engineering researcher 

  • the particular research that Silvia is involved in

Facilitators themselves can also act as a great source of evaluation data. We encouraged our ECRs to take an active part in evaluations, and some theme teams specifically requested that the ECRs on the team write up the activity reports. Two main evaluation methods were used:

  • an Online Questionnaire
  • a Reflective Log.

The online questionnaire was used to help us evaluate the coaching days we provided for ECRs:

Coaching and mentoring training - course attended 24th June

You have been sent this short survey as you attended a Coaching and Mentoring course on 24th June 2014 at St. Luke's Campus, University of Exeter.

In order to evaluate the course, and improve participant experience we would like to ask you a few questions about the course. There are 6 short questions in total, 3 of which are optional. Responses are anonymous.

Thank you for participating in our survey. Your feedback is important.

Q1: How confident do you feel in interacting with young people about your research AS A RESULT OF THIS COURSE?

  • More confident
  • The same as I felt before the course
  • Less confident

Any comments about confidence in interacting with young people about your research:

Q2: How knowledgeable do you feel about the current school context in England AS A RESULT OF THIS COURSE?

  • More knowledgeable
  • Neither more or less knowledgeable
  • Less knowledgeable

Any comments about your knowledge of the current school context:

Q3: How aware are you of the Widening Participation agenda AS A RESULT OF THIS COURSE?

  • More aware
  • Neither more or less aware
  • Less aware

Any comments about your awareness of the Widening Participation agenda:

Q4: Something I learnt in this course was:

Q5: If asked to explain my research to a 14-year-old I would say:

Q6: Do you have any other comments or questions about this course?

 

Finally we encouraged ECRs to use the Reflective log document following an event or activity, to help reflect on their experiences.

 

There are a host of excellent evaluation resources available on this website