A quantitative study into feedback strategies and the impact upon metacognition
The Education Endowment Foundation (EEF) marking review report (Elliott et al. 2016) highlighted two areas where there was a paucity of research: ‘Testing the impact of marking policies which are primarily based on formative comments and rarely award grades’ and ‘Investigating the most effective ways to use class time for pupils to respond to marking’. This piece of action research attempts to ascertain if marking without grades, and providing feedback with a grid and comment, leading to directed improvement and reflection time, creates a quantifiable improvement in students’ metacognitive abilities.
Metacognition concerns not just what students are learning, but how they are learning (i.e. how they learn best by ‘learning to learn’ (Quigley and Stringer 2018)). Training students in metacognition creates ‘more awareness and understanding among pupils to help them engage and embed their learning more effectively’ (Mughal 2018). Developing students’ metacognitive abilities is thought to increase progress by an additional seven to nine months (Evidence Based Education 2018; Lockyear 2018). Assessment for learning (AfL) involves formative feedback. It identifies where students are in their learning, where they need to get to, and how to get there (Wiliam 2011). AfL is known to ‘substantially increase student attainment’ (Elliot et al. 2016). One strategy to demonstrate to students how to progress learning is the provision of assessment criteria prior to completing an assessment. This is thought to have a positive impact in assisting student understanding of task requirements (Bell et al. 2013).
Assessment criteria grids can be particularly beneficial for providing feedback. Methods to help students to engage with feedback to assist their metacognition include ‘closing the gap’ marking where the teacher puts a double tick next to a good point, or writes coded feedback such as WWWT? (what’s wrong with this?) or RTQ (read the question). The student then works to identify why this specific feedback was given and seeks to improve (Fletcher-Wood 2013), therefore engaging more than they would having simply read written feedback. Coded feedback encourages students to self-correct rather than relying on having mistakes corrected (Bates 2016). Interestingly, research suggests there is no discernible difference in outcome between providing coded or lengthier written feedback, so long as students understand the codes (Elliott et al. 2016).
A quantitative study into feedback strategies and the impact upon metacognition Another method to improve student engagement is not providing a numerical grade as ‘grades stop learning’ (Wiliam 2011). It is argued that feedback with grades dictates a greater level of ego involvement and competition with students competing with peers for the better grade (Woolfolk et al. 2013), as opposed to greater task involvement from just giving comments, which students have to focus on more thoroughly (Wiliam 2011). This sense of competition can be particularly detrimental for anxious pupils, those lacking in self-confidence, or those less prepared for assessments (Woolfolk et al. 2013). The EEF marking review (2016) recognises that students will not benefit from feedback unless time is given to respond to marking. Such time has various different names – ‘feed forward time’, and ‘directed improvement and reflection time (DIRT)’ (Kay et al. 2016). Studies suggest it is valuable to give students DIRT time in class to engage with and respond to feedback (Elliott et al. 2016).
In their first Geography lesson of the academic year 2018-19, all Year 8 geography students undertook a ‘Learning and Study Strategies Inventory’ (LASSI) questionnaire (H & H Publishing 2018) to test their baseline study skills. LASSI is noted as a ‘statistically valid and reliable tool for the diagnosis of study skills’ and is a useful tool for evaluating intervention strategies (Weinstein et al. 2018). The LASSI questionnaire consists of questions testing: anxiety, attitude, concentration, information processing, motivation, selecting main ideas, self-testing, test strategies, time management, and using academic resources. A t-test established that there was no significant difference at baseline testing (LASSI-1) between the intervention and control groups in any of the LASSI component variables.
Elen Harris’ 8X class was used as the intervention set, whilst her 8F class was used as a control with a ‘business as usual’ approach to feedback. ‘Control’ group feedback on assessed work included comments written on student work, with an overall comment and grade. Intervention students were provided with feedback on a standardised template, designed after reviewing existing literature. Key features included: a marking criteria grid, and space for teacher comment and student feedforward targets. Importantly, no grades were provided.
The template works by the teacher highlighting the relevant marking criteria met by the student and adding coded feedback to their work. Six intervention group students were part of a focus group to gain their opinions on feedback strategies.
The LASSI questionnaire was repeated (LASSI-2) after the last assignment of the year. Maths Teacher, Paul Parham, then completed statistical analysis of both the baseline and post-intervention LASSI data to ascertain if the DIRT feedback techniques had a quantifiable impact upon student metacognition.
Findings and discussion
Method of feedback
The feedback grid is easy to use and quick for the teacher to mark and add a comment. Comments occasionally repeat the grid so must be meaningful. Marking codes are time efficient, though need to be adapted for each assessment, and a bank of codes can be accumulated over time. Focus group students liked the method of feedback, though it took a couple of reflections to get used to this style. As a consequence, the intervention class fell slightly behind other classes in terms of content delivery. This could be rectified by re-planning lesson content and students would become quicker at completing as familiarity increased.
In the focus groups, interestingly, male students stated they would prefer to receive a grade as they believe it demonstrates where they are in their learning and how much they need to do to improve, with the potential to track progress. Conversely, female students liked not having a grade; they believed this lessened stress levels by reducing competition in class and made them focus more on the feedback and assessment criteria. Students praised the provision of marking criteria and found it incredibly useful to see exactly what they needed to do for an assessment. Interestingly, they viewed their involvement in the creation of assessment criteria as a waste of time, as they could be getting on with the assessed piece of work instead.
The students viewed spending class time on DIRT as pointless, stating they were simply reiterating points the teacher had already made. They naturally review feedback, undertake self-assessment, and think about how to improve anyway. Students who volunteered for the focus group are all high achieving, self-motivated students who are perhaps more likely to do this than lower ability students (and therefore this might be an anomaly). When asked whether different teachers or departments having different approaches to feedback was difficult, students did not perceive this as a problem as long as feedback was clear and consistent students could manage different feedback mechanisms.
The t-test looking at boys and girls mean scores revealed that there was no significant difference between boys’ and girls’ overall scores in LASSI-1 or LASSI-2, nor in any individual metacognition components, apart from the use of study aids in which girls performed significantly better than boys (boys mean score of 20.93 (standard deviation (SD) of 4.72) and girls 24.22 (SD of 5.03) in LASSI-1 leading to p = 0.003, and boys 20.37 (SD 5.06) and girls 24.16 (SD 5.01) leading to p = 0.001 in LASSI-2). It could be said that boys should therefore be encouraged more to utilise study aids in their work, though the results are not statistically significant enough to conclude that boys should be given more support in this area and again this would need confirming via another study.
Interestingly, the t-test also demonstrated that there was a significant difference in anxiety levels between boys and girls in LASSI-2 (boys mean score of 30.08 (SD 7.54) and girls 22.43 (SD 7.37) leading to a p-value of <0.005) with boys performing better than girls. Although boys had performed better than girls in LASSI-1 (boys 27.38 (SD 7.45 and girls 24.43 (SD 6.87) this was not significant at the 5% level (p = 0.066) suggesting girls’ anxiety had got worse throughout the academic year. This was investigated further using a paired t-test and it was found that boys performed significantly better in dealing with anxiety between LASSI-1 and LASSI-2 (p = 0.007) while girls showed no significant difference (p = 0.072). Hence, although girls’ ability to cope with anxiety had got worse as the year progressed, this was not significant at the 5% level (though it did approach statistical significance) and the reason for the gap widening was down to boys coping better in this instance.
When using a t-test to analyse LASSI components longitudinally between LASSI-1 and LASSI-2 there is no significant difference at the 5% level between component scores, suggesting that the intervention strategy had no impact upon students’ metacognitive abilities. Despite this, in the intervention group the two biggest changes approaching statistical significance over time were in attitude (p = 0.074) and test strategies (p = 0.089). It could be inferred that the DIRT approach did have some impact on these components of metacognition, though the result is not significant enough to firmly draw this conclusion and students would be exposed to many other feedback methods in the school which could have affected this. In addition, multivariate linear regression analysis demonstrates that LASSI-2 scores were not affected by whether a student was in the intervention or control group, further giving evidence that the DIRT template intervention did not have an impact.
The multivariate linear regression analysis gives further evidence to the t-test results for differences between boys and girls, as gender was found to significantly affect LASSI-2 scores in anxiety, attitude, concentration, information processing, motivation, and time management, as well as the total LASSI-2 score. Beta coefficients at 95% confidence intervals demonstrate that girls’ scores are less than boys’ scores after adjusting for the effect of LASSI-1 total scores, whether the student was in the intervention or control group, nationality, age, and distance from school. The largest effect for a single metacognition component was the adverse effect on anxiety (ß= -7.656 with a p value of <0.005). This is in contrast to the focus group indicators with respect to anxiety. It was also found that distance from school significantly affected LASSI-2 scores in anxiety, selecting main ideas, and time management, alongside the total LASSI-2 score with increased distance leading to a higher component score. For example, for each additional mile away from school, the anxiety score increased by 0.181 (p = 0.010). However, the magnitude of this effect is very small and far less influential than gender.
This study is on a small scale over a short time period; a larger scale, longitudinal study is recommended to test whether DIRT strategies do have any impact upon metacognition over a longer time period. The metacognitive benefits may not be clear at the end of the year, but will be apparent in sustained results in subsequent years (Lockyear 2015). It was difficult to fully attain results over just one academic year with a relatively assessment light curriculum and it is suspected that over a longer time period, with a more cohesive approach involving other departments, DIRT would have an impact. This recommendation is difficult to achieve in Sevenoaks due to students moving classes each year and department approaches to feedback differing. It would also be interesting to further study the impacts of anxiety and sleep etc. on student metacognition. If these further studies were to occur and had statistically significant findings then it would be recommended to include elements of metacognitive training into the PSHE curriculum (for example, to aid students in these areas).