Correlations of CAT4 and GCSE grades
As already stated, the strength of the relationship between two variables can be measured by a statistic called the correlation coefficient. A value of zero indicates no relationship between the two measures, whereas a value of one indicates a perfect positive relationship. The table below shows the correlation coefficients between CAT4 standard age scores and pupils’ subsequent GCSE outcomes.
The correlations are all highly significant. Most GCSE outcomes tend to have their highest correlation with mean CAT4 score. The exceptions are English Language and English Literature where the CAT4 Verbal Reasoning score gives a slightly higher correlation than mean CAT4 score.
Likelihood of GCSE indicated grades
The example below illustrates the probabilities of achieving the various GCSE 9-1 grades in Mathematics (U is ungraded) for a student with a mean CAT4 score of 100. The indicators are not precise: they indicate the outcomes expected for students with a particular CAT4 score making average progress in a typical secondary school.
The ‘most likely grade achieved’ is reported to one decimal place. In this case the student is expected to be on the top end of grade 4 as he has a 52% chance of achieving grade 4 or below and a 48% chance of achieving grade 5 or above, so the expectation is that the student is near the grade 4/5 boundary.
The example below illustrates the probabilities of achieving the various GCSE A*-G grades in History (U is ungraded) for a student with a mean CAT4 score of 100.
The ‘most likely grade achieved’ is grade C with the student having a 64% chance of achieving grade C or below and a 34% chance of achieving grade B or above.
GCSE grade indicators for groups of students
The table below illustrates how the group/class indicators have been calculated for a fictitious class with five students and shows the most likely grade achieved and the probabilities associated with getting different Mathematics 9-1 grades. The group indicator is an average of the individual student outcomes and probabilities. A similar method is used for subjects using the A*-G grades.
Using individual student grade estimates to provide information about the overall class or group grade outcomes will in most cases lead to underestimating the number of students likely to get both the higher and lower GCSE grades.
The group level indicators are the average of the probabilities for all students in the group. Our research has shown that this method provides the most accurate set of group level indicators. However, group indicators are extremely sensitive to variations in the number of students in the group, and may be very unstable for groups of less than 30 students. Group indicators should only ever be taken as a rough guide to the possible future performance of a class.
CAT4 and GCSE Attainment 8
The graph below illustrates the relationship between CAT4 score and the Attainment 8 score.
For example, for a student with a mean CAT4 score of 90, the most likely Attainment 8 is 42 and the ‘if challenged’ score is 49. Not all students with a mean CAT4 score of 90 will get an Attainment 8 score of 35.
Around half the students will get an Attainment 8 score below 35, with around 25% of the students obtaining an Attainment 8 score of less than 26 – the bottom 25th percentile. Around 25% of students will obtain the 'if challenged’ score of 43 and above.
CAT4 and GCSE Grades A* - G
Wales is retaining the current A*-G grading system; but in Northern Ireland the GCSE grading system is currently the same as for England, using the mixture of A*-G and 9-1 grades. A new structure based on a revised A*-G grading system was implemented in Northern Ireland in summer 2019. The new A* aligns closely to grade 9, and a new C* grade is equivalent to grade 5.
The graph below illustrates the proportion of students achieving five+ GCSE grades 9-4 (A*-C) including English and Mathematics for each mean CAT4 score. We can see that the higher the mean CAT4 score, the greater the proportion of students who achieve five or more A* to C grades. For example, only 17% of students with a mean CAT4 score of 85 obtain five+ 9-4 (A*-C) grades; in contrast, about 89% of students with a mean CAT4 score of 115 achieve five+ 9-4 (A*-C) grades.
The above confirms the need for suitably cautious interpretation when using the indicators with staff and parents, and particularly if sharing them with individual students. In the latter context, we would advise that school staff follow the established best practice of schools, using the results for mentoring and target-setting purposes by:
- stressing to students that the indicators are a statistical prediction, not a prophecy of their actual Leaving Certificate, Key Stage or GCSE results;
- emphasising to students the range of outcomes that could be achieved;
- emphasising the importance of the students’ motivation and effort in determining the grade they obtain, identifying any areas in which the student requires greater support from the teacher;
- not using the indicators to label students as actual or potential ‘failures’;
- setting the indicators in the context of all other known relevant factors and other assessment information, thus making sure targets are reasonable.