Case Study 1: CAT4 at transition

The short case studies in this section show how CAT4 is used in schools with both groups and individual students. The cases are drawn from the very large number of schools that took part in the standardisation of CAT4 and then received full reports based on the published data.

An overview of how CAT4 may be used to look at groups of students, plus some contrasting individual profiles, is given in the case studies for two secondary schools. The primary school case study (Case study 3) incorporates some issues schools may want to consider when testing children for whom English is an additional language. It also describes three students who have similar profiles but where contextual information plays an important part.

This case study is taken from a mixed 11 to 18 comprehensive school in central England. It is a Specialist Science and Language College and also a training school. The school roll is currently around 1,700, with this expected to increase to around 1,900 in the near future. The school places a strong emphasis on both the academic and personal development of its students. Its academic results have been consistently high and, in 2008, it was awarded an Ofsted ‘outstanding’ rating

The school is ethnically very diverse, with around 40% of students from minority ethnic backgrounds. This is a much higher proportion than normal for the area and is largely due to it being the only faith school in the area, so it attracts a large number of students from Eastern European backgrounds. The school has a lower than average percentage of students entitled to free school meals, although students come from a wide range of social and economic backgrounds.

CAT4 is given to all Year 7 students in their first week at school. The Head of Learning Support explained the importance of getting CAT4 results back as soon as possible, so that she can use them as part of the identification of students with specific support needs and also ‘gifted and talented’ students.

CAT4 results are not currently sent to all teachers, but results are reviewed by the Head of Learning Support and used to identify the following students:

• The lowest scoring students in each year group (in conjunction with information from feeder schools): These students enter a small class of around 10 students where they are given intensive support. This class has a single teacher for around 50% of the time, and so replicates the feel of a primary school in some respects. Depending on progress, students may move out of this class into the mainstream classes, although others may remain in the class for the duration of their time at school.
• Students with stanine scores of 1 or 2 on the Verbal Reasoning Battery who may have reading difficulties: These students are then screened using a Reading test that produces further diagnostic information about their reading abilities. Where further testing reveals that students do need support in their reading, a plan for this is then put in place. There is currently discussion about whether the range of verbal scores used to identify students for further screening should be extended to stanines of 3 and below.1
• The lowest attaining students at the point of entry to the school, based on those reported as operating below Level 2 by the feeder school and their CAT4 results: These students are enrolled on a programme of intense literacy support and are not entered to study a foreign language.
• Any students with an SAS of less than 90 for the Quantitative Reasoning Battery: These students are made known to the Head of Maths. In evaluating students’ needs in the area of Mathematics, CAT4 results are integrated with the results from other assessments, but they do serve as an early indicator of students who may need additional support.

In addition, KS2 and CAT4 results are used together to set KS4 (GCSE) attainment targets for all students. Government targets specify that three levels of progress are expected between KS2 and KS4. In accordance with the school’s ethos to support children in achieving at their very best, more challenging targets are set that represent four levels of progress from Year 7 to GCSE.

Sometimes KS2 data can be inaccurate if a student was not entered for the Year 6 SATs. A teacher-assessed level is often submitted instead. If this is the case, the school tends to start these students at NC Level 1 as a baseline at Year 7 and adjust accordingly once subject assessments have been completed. The CAT4 data are also used as they give the school an overview of the student’s strengths and weaknesses. This is then cross-referenced with subject assessments – for example, a high score on the Quantitative Reasoning Battery should indicate that the student will be in a higher Maths set.

If there is a discrepancy between KS2 and CAT4 data, the school tends to give more weight to the KS2 score as this reflects acquired learning over time rather than a ‘snapshot’ of what they did on the day.

Detailed examples of how the school uses CAT4 results for individual students are given below.

1 Recommendations contained in the Individual report for teachers are that a stanine score of 3 or below on the Verbal Reasoning Battery should trigger an assessment of a student’s reading.

Daniel is a student with English as his first language. He reached the expected age milestones and, by the end of KS2, Maths and Science. His Literacy, however, has always needed additional support as he has had an ongoing hearing problem (‘glue ear’). Daniel’s strengths lie in his visual learning as he has had to develop this to compensate for his earlier difficulty with hearing. Although he is now within the average range for Literacy, Daniel works better when information is presented visually.

Daniel’s CAT4 scores are:

Daniel’s table of results shows that he completed all questions on each of the CAT4 batteries, with the exception of the Nonverbal Reasoning Battery where just one of the 48 questions was not answered. Given this, and that all his raw scores are above the chance level, we can be confident that Daniel’s profile is likely to be a reliable reflection of his abilities.

Daniel’s profile shows his nonverbal ability, and particularly his spatial ability, to be stronger than his verbal and quantitative abilities.

• His verbal SAS is 97, which is equivalent to a stanine of 5. Daniel’s percentile rank is 42, showing that he performed as well as or better than 42% of the national sample. This level of performance would be described as within the average band.
• Daniel’s quantitative SAS is 105, which is equivalent to a stanine of 6. His percentile rank is 63, showing he scored as well as or better than 63% of the national sample. This level of performance would be described as within the average band.
• His nonverbal SAS is 121, which is equivalent to a stanine of 8 and a percentile rank of 92. This level of performance would be described as within the above average band.
• Daniel’s spatial SAS is 129, which is equivalent to a stanine of 9 and a percentile rank of 97. This level of performance would be described as within the very high band.
• Daniel’s mean SAS of 113 indicates that he is performing at an at least average level in all areas.

An examination of the confidence bands shows the areas of relatively higher and lower performance for Daniel. When looking at differences between scores on the different CAT4 batteries, it is important to pay attention to the confidence bands. No measurement of abilities is perfect and all contain a degree of error. This error is reflected in the confidence bands, which describe the range within which we can be reasonably certain – in the case of CAT4, 90% certain – that Daniel’s ‘true score’ on each battery lies. In this context, true score refers to the score Daniel would achieve if the measurement was completely free of error. Using confidence bands appropriately ensures we do not over-interpret small differences between scores on different batteries, leading us to conclude that differences in performance exist when in fact they do not.

Starting with Daniel’s strongest score, which was obtained in the Spatial Ability Battery, we can see that the confidence band for this battery does not overlap with the confidence bands for the Verbal or Quantitative Reasoning Batteries. We can therefore be 90% confident that Daniel’s spatial ability is significantly stronger than his verbal or quantitative abilities. Daniel’s second highest score was on the Nonverbal Reasoning Battery. The confidence band for the Nonverbal Reasoning Battery does not overlap with the confidence band for the Verbal Reasoning Battery, so we can be at least 90% confident that there is a significant difference between his performance on these two batteries.

Looking at the Nonverbal and Quantitative Reasoning Batteries, we see that there is a very small overlap between the confidence bands. Even though this overlap is very small, it does mean that we cannot be 90% confident that there is a real difference between these two abilities. As the 90% confidence bands for his verbal and quantitative scores overlap, it can be concluded that there is no significant difference in Daniel’s reasoning abilities in these two areas.

His CAT4 mean overall SAS of 113 shows Daniel to be performing at at least an average level in all areas. This mean value is found by summing the SAS for each of the four batteries and dividing by four (that is, the number of batteries in CAT4). As Daniel’s performance varies considerably across the four CAT4 batteries, his overall score should be interpreted with a degree of caution. Although the overall SAS gives a general indication of his combined reasoning abilities, it does mask considerable variations in Daniel’s performance across the four batteries.

The difference between Daniel’s verbal and spatial scores of 4 stanine points shows an extreme bias towards spatial processing. This is described in his narrative profile summary as follows:

• This profile demonstrates a distinct relative strength in spatial over verbal learning.
• Daniel should excel when engaged in tasks that require visualisation and will learn well when working with pictures, diagrams, 3D objects, mind maps and other tangible methods.
• Relatively weaker verbal skills may make learning through written texts, writing and discussion less effective.
• Daniel is highly likely to enjoy and learn best through active learning methods such as modelling, demonstrating and simulations and should be encouraged to problem-solve and develop his own ideas through these methods.
• Daniel should do very well in subjects that make the most of his spatial ability, such as Science, Technology, Design and Geography but will find language-based subjects such as English, History and Modern Foreign Languages less rewarding unless teaching methods are adapted to suit his profile.

The Individual report for teachers also includes narrative on the implications for teaching and learning, as described here:

• Daniel has a very strong understanding of spatial concepts, with average verbal reasoning skills.
• Students with such high levels of spatial ability are often characterised as ‘intuitive’ and as those who see the ‘bigger picture’. This can be at the expense of a lack of attention to detail which may be characteristic of Daniel.
• Daniel should be encouraged to explain his understanding of spatial activities and reflect critically upon them to further enhance his verbal reasoning skills.
• Placing Daniel in paired work with others, perhaps with higher level verbal skills, could provide mutual benefits.
• Daniel may perform better where spatial and visual approaches to learning are used. For example, enacting scenes from a Shakespeare play can provide strong visual images that will help in written composition.

From the CAT4 data, Daniel is predicted to achieve A* at GCSE level in subjects such as Design and Technology, Science and Geography. The more literacy based subjects such as English, RE and History predict B at GCSE level. Knowing where his strengths lie would inform his teachers about his learning preferences and ensure that strategies used throughout KS3 and KS4 are adapted to suit his needs. With these in place Daniel has every opportunity of achieving A* in every subject.

Damian is a student with English as an additional language. He has an innate love of learning and has excelled at everything he has done throughout KS1 and KS2. Damian developed early literacy skills and was a fluent and expressive reader by the end of Year 1. He speaks English and Tagalog (Filipino) at home and, although classed as an ‘EAL learner’, this has had little impact on his learning. Damian has developed sophisticated language skills and this is reflected in his writing, which is enticing and captivating, with an immense imagination that engages his audience.

Damian’s CAT4 scores are:

Damian’s table of results shows that he completed all questions on each of the CAT4 batteries, with the exception of the Nonverbal Reasoning Battery where 47 out of the 48 questions were completed. Given this, and that all his raw scores are well above the chance level, we can be confident that Damian’s profile is likely to be an accurate indication of his abilities.

At first glance Damian’s profile shows a highly differentiated pattern of results, with his strongest score being in the area of verbal ability, followed by quantitative, nonverbal and then spatial.

• Damian’s strongest score is in the area of verbal reasoning, where he achieved an SAS of 141. This is equivalent to a stanine of 9 and a percentile rank of 100, meaning he is in the top 1% of the national population. This level of performance would be described as within the very high band.
• Damian’s quantitative SAS is 125, which is equivalent to a stanine of 8. His percentile rank is 95, showing he scored as well as or better than 95% of the national sample. This level of performance would be described as within the above average band.​
• His nonverbal SAS is 113, which is equivalent to a stanine of 7 and a percentile rank of 80. This level of performance would be described as within the above average band.
• Damian’s spatial SAS is 98, which is equivalent to a stanine of 5 and a percentile rank of 45. This level of performance would be described as within the average band.
• Lastly, Damian’s mean SAS of 119 indicates that he is performing at an above average level across all areas.

Damian’s strongest score is on the Verbal Reasoning Battery. An examination of the 90% confidence bands shows that the confidence band for the Verbal Reasoning Battery overlaps with the confidence band for the Quantitative Reasoning Battery but not with the confidence bands for the Nonverbal Reasoning Battery or Spatial Ability Battery. This indicates that his verbal ability is stronger than his nonverbal and spatial abilities, but that there is no significant difference between his verbal and quantitative scores. The confidence band for Damian’s Quantitative Reasoning Battery score overlaps with the confidence band for his Nonverbal Reasoning Battery score but not his Spatial Ability Battery score. This indicates a stronger performance in quantitative than spatial reasoning, but no significant difference between his quantitative and nonverbal abilities. Similarly, as the confidence bands for his Nonverbal Reasoning Battery and Spatial Ability Battery scores overlap, there is no significant difference in his performance in these areas.

His CAT4 mean overall SAS of 119 shows Damian to be performing at an above average level across all areas. However, even more so than the profile of Daniel described above, his performance varies considerably across the four CAT4 batteries. Again, this means that while his overall SAS gives a broad indication of his reasoning abilities, it should be used cautiously.

The difference between Damian’s spatial and verbal scores of 4 stanine points shows an extreme bias towards verbal processing. This is described in the narrative profile summary as follows:

• This profile demonstrates a distinct strength in verbal compared to spatial learning.
• Damian should excel when engaged in tasks that make the most of his very strong verbal skills, including learning through written texts, writing and discussion.
• Relatively weaker spatial skills – which are, however, in the average range – will make learning through visualisation, working with pictures, diagrams, 3D objects, mind maps and other tangible methods less attractive. With encouragement, these methods can make learning more engaging and effective for Damian.
• Damian is highly likely to enjoy and learn best by talking about learning, ideas and opinions, gathering information through reading and through both factual writing and creative writing tasks.
• Damian should do very well in subjects that make the most of his verbal ability, such as English, Humanities and Modern Foreign Languages.
• Damian may find certain aspects of subjects such as Science, Technology, Design and Geography less rewarding unless teaching methods are adapted to suit his profile or his spatial skills are developed to more closely match his verbal skills.

The Individual report for teachers also includes narrative on the implications for teaching and learning, as described here:

• Wherever the understanding of spatial concepts is required in the curriculum, such as Art, Design, Science and Maths, teachers should be aware that Damian may require some additional support.
• However, given his excellent verbal reasoning skills, expectations need to be appropriately high, with enrichment activities to provide challenge and extension.
• While teachers should continue to use a broad and varied range of styles, it is likely that Damian will be a self-motivated and independent learner.
• Teachers should encourage Damian to follow his interests, and he will benefit from a fast pace of instruction, tend to learn very quickly and respond well to tasks that develop his independent study skills.
• Extension activities that require him to form hypotheses, make predictions and test outcomes may be particularly helpful.
• Q&A sessions should be used to develop higher order thinking skills by requiring Damian to justify opinions.
• Damian should be encouraged to read extensively and choose from a wide range of material.
• Damian may enjoy creative writing and discussion and debate and should be encouraged to develop such interest, both in lessons and through extra-curricular activities.

Information gathered from the CAT4 data is invaluable in identifying those at the top end of the year group and in ensuring that teachers are aware of the students’ needs as well as providing opportunities for challenging and extending learning. It offers information for setting up ‘target groups’ with like-minded students that can focus on study techniques, revision and exam preparation. The lowest scoring students in the year group can be identified easily and further assessments administered to establish appropriate interventions. The key message is ensuring staff are made aware of the identified students and that they are trained in appropriate differentiation of materials for the high and low ability students that cater for their specific strengths and weaknesses. When the students are exposed to this within teaching and learning they have every opportunity to make the expected progress, if not more.