Technical Report Information
Trialling and standardisation
The sentence completion, reading comprehension and phonics questions for the New Group Reading Test (NGRT) were developed by English subject experts.
These questions were initially developed for the paper version of NGRT. They were trialled in 2009 using 2,481 students in 73 schools. After the trials seven paper tests were developed for standardisation for students ages 5 to 16 years. The paper standardisation was conducted in the United Kingdom and took place in 2010 using a total of 11,716 students from 476 schools. A further four paper tests were developed and trialled in 2013 using 1,635 students in 34 schools. The standardisation for these new paper tests took place in 2014 using a total of 4,960 students from 187 schools. The reliability of every paper test is above 0.9 which is high. More details of the trial and standardisation of the paper tests are given in the NGRT paper Teacher Guides.
The digital version of NGRT uses the same questions as those in the paper version. Three parallel versions of the standardised tests (Forms A, B and C) were created, with questions of similar levels of difficulties in each form. The individual question statistics, derived from the paper trials and standardisation, were used to map all the reading questions across all the booklets onto a common difficulty scale, and estimates of an overall reading ability were derived for the paper versions of the tests. The standardised scores for the digital version are based on the reading ability scores. Item Response Theory was used to derive the common difficulty scale and model the adaptive method used to select questions in the digital version of the test.
Paper and Digital Equating study
To verify the digital version, an equating study with students doing both the paper and the digital versions of the tests was carried out in 2012 to see how the scores compared. The results showed that the digital test scores for 6 year old students (Year 1 in England and Wales) were much lower than for the paper version. It may be that the students at this age were having difficulty with using the digital version of the test; for this reason we recommend that the digital version of NGRT is not used by Year 1 students; the youngest recommended year group is Year 2.
There were 1721 students in Years 2 to 10 who did both the paper and digital versions of the test. The adjusted correlation between paper and digital versions after correcting for measurement error was 0.91. Overall digital scores were slightly lower than paper versions; adjustments were made to the scores of the digital version of the test to account for this difference.
Overall females perform better than males by an average of 3.1 SAS points. This gender difference is significant and females in general, on average, tend to perform better than males at all age groups in primary and secondary schools.
Overall non-EAL students perform better than EAL students by an average of 3.6 SAS points. This difference is significant and the differences exists at all age groups. EAL on its own does not give an indication of a student’s proficiency in the English language as other contextual factors such as how long the student has been in the country or the age of the student on arrival in the country etc. also need to be considered.