Learning from the pilot cohort of the Levelling Up Widening Participation Programme
Dr Helen Cramman, School of Education, Durham University and Head of the Evaluation and Innovation in Science Education (EISE) Research Group writes a guest blog summarising her recent research report.
The first cohort of students on the Levelling Up Widening Participation Programme completed their time on the programme in Spring 2022 and the evaluation of their experiences has recently been published. The evaluation found that the programme had successfully accomplished the aims it set out to achieve at the beginning of the programme, and also presented some interesting and important suggestions to aid future implementation.
What is the Levelling Up programme?
The Levelling Up Pilot Widening Participation Programme is a national academic and pastoral online support programme spanning two academic years, targeted at Year 12 students in England who have an interest in potentially pursuing the study of Chemistry, Maths or Physics at University. There were six Spokes in the pilot programme, with three Hubs coordinating the overall running of each subject strand. Durham University led the Chemistry strand (1 Spoke), the London Mathematical Society led the Maths strand (2 Spokes), and the Institute of Physics led the Physics strand (3 Spokes). The pilot cohorts started the programme between February 2021 and July 2021 and the last sessions ran between March 2022 and June 2022. In total, 226 students accepted places on the pilot cohort of the programme. Each programme had a different combination of activities for the students.
| Chemistry - Durham | Maths - Durham | Maths - Leicester | Physics - Birmingham | Physics - Durham | Physics - Oxford | |
|---|---|---|---|---|---|---|
| Subject specific tutorials | Yes | Yes | Yes | Yes | Yes | Yes |
| Separate mentoring sessions | Yes | No | No | Yes | Yes | Yes |
| Guest lectures | Yes | Yes | Yes | No | Yes | Yes |
| Onsite/remote university visits | Yes | Yes | Yes | No | Yes | No |
| Guaranteed conditional university offer | Yes | Yes | Yes | No | Yes | No |
| Graphics tablets provided to students | No | Yes | Yes | No | No | No |
Defining success criteria
Before the Levelling Up programme started, a detailed Theory of Change model was created in collaboration with the Chemistry, Maths and Physics Hubs. This identified what the programme sought to achieve (what impact did it hope to have), what outcomes did it need to happen in order for those impact aims to be met, what activities would help achieve those outcomes and impact and importantly, what assumptions were being made in the design of the programme. Using the Theory of Change model, it was then possible for the evaluation to design a series of measures that would identify if those impact aims and outcomes had been successfully achieved. The Seven aims for the Levelling Up Pilot Programme were:
- Participants aspire to study chemistry, physics, mathematics, or a directly related STEM discipline to their programme subject, at university.
- Participants apply to a high ranked university as listed in the Times Good University Guide
- Participants aspire to study at their Levelling Up host university.
- Participants aspire to study at university (in any subject).
- Participants consider that the programme has helped them achieve higher grades at A level in their subject.
- (Chemistry and physics) Students consider that the programme has helped them achieve higher grades at A level in maths within their subjects
- Participants received offers to study the courses which they have applied for on their UCAS applications
The evaluation collected information at multiple timepoints across the full duration of the students’ time on the programme. This included application form data, surveys, interviews and focus groups and observing training sessions for tutors and mentors. Permission from participants was sought for their inclusion in the evaluation and the use of their data.
Were the activities delivered as planned?
The first question to be asked by the evaluation was whether all activities had been delivered as planned. The findings showed that, yes, to some extent, all activities were delivered as planned. There were, however, variations in delivery across Spokes as well as in the engagement from participants.
Did the expected outcomes occur and were the impact aims achieved?
The next question was to look at whether the expected outcomes and impact aims had been achieved. The evaluation found evidence that the intended outcomes had taken place, with four out of the six Spokes meeting all the impact aims. However, achieving the aim of participants applying to a high ranked university as listed in the Times Good University Guide was slightly weaker for the Maths – (Leicester) and Physics (Birmingham) Spokes. Maths (Leicester) and Physics (Oxford) did not meet the aim of students applying to their host university, however, this was not of concern to the two Spokes as this was not a driver within their university widening participation remits.
Could anything else have impacted on the programme?
The final question to investigate was whether there were any external contextual factors that may have negatively affected the intended outcomes or impact for the programme. None were found from the evidence collected. However, the data did highlight the complexity of differing needs of the students.
What was the final conclusion?
Based on the evidence collected, the final conclusion from the evaluation was that it is reasonable to conclude that the Levelling Up programme contributed to achieving the stated impact aims for the programme.
Five recommendations for future refinement of the programme were made:
- tutoring and mentoring provision should run with small group sessions (max 6 students), with the same students, tutors/ mentors in a group week on week;
- the opportunity should be provided for students to communicate with each other outside the organised weekly sessions;
- support should be provided to tutors and mentors in tailoring topics and differentiating difficulty of activities for the interests and needs of students within their group;
- graphics tablets should be provided for all subjects to support drawing graphs and writing equations;
- and best practice should be considered for most effectively working with students that are not visible on screen (e.g. camera off) or audible (e.g. microphones off).
For more information, please contact the lead evaluator, Dr Helen Cramman, at levelling.up@durham.ac.uk The full report can be downloaded at https://collections.durham.ac.uk/files/r18910jt623#.Y8lrcXbP2Uk