Guidelines for test administration
Trial run-through including how to exit during a test
Assessing students with Exact is straightforward but before the teacher or administrator attempts to test any student it is advisable first to run through the complete suite of tests to familiarise themselves thoroughly. Ideally, this should be done in the same or similar test situation in which the students will be (see Section 2.6.3). To do this, register yourself as a student.
Testing environment and equipment
The ideal testing environment is one that is reasonably quiet, with minimal distractions. Ideally, this should be a separate room, but Exact has been designed to be robust for use in the ordinary classroom, provided visual and auditory distractions (both to the student being tested and to other students in the class) have been minimised. To minimise auditory distraction, headphones are recommended. Inexpensive lightweight headphones of the type used for portable audio equipment will be adequate (but not the type that are inserted into the ear). Teacher or supervisor judgment is paramount in ensuring the appropriate testing environment.
If assessment is going to be carried out in an ordinary classroom in which there are other pupils, the computer and the student should be positioned in such a way that the student is not looking directly at the rest of the class, nor should the rest of the class easily be able to see the monitor screen. The best position for this is usually in the corner of the room. Students should not attempt the tests when other students are in a position in which they can become involved in the task or act as a distraction. It would be hard for other students to inhibit their reactions, and their behaviour could influence the decisions of the student being tested.
The teacher or supervisor should check that the equipment being used for the assessment is functioning correctly. This includes checking:
- that the sound system (speakers or headphones) is audible (not too loud or too soft, and without interference); and
- that the mouse is functioning correctly (non-optical types, particularly, require regular cleaning) and is positioned in front of the student on a suitable surface so that its movements are unimpeded. Please note that Exact should be used with a mouse (wired or wireless), not a touchpad as this will affect response times.
Exact should not be used for testing when any other applications are running on the computer, as these can interfere with the timings and recording of results. Please close down all other applications before starting Exact.
Before testing, each student must be registered on Testwise. See Section 1.8 for guidance on this. The tests can be done in any order but it is usually best to start with word recognition which students generally find quick and easy. Instructions are spoken by the computer, and each test commences with a practice or demonstration of the task. When the student has completed the practice items, the test phase begins.
The student should be sitting comfortably at a suitable level in front of the computer screen (not too high or low, in order for them to see the screen and use the mouse satisfactorily). It is not recommended that students attempt the tests standing up, as they are more likely to move about and alter the angle at which the screen is viewed – this can lead to failure to see everything that is happening on the monitor and can also disrupt mouse control. The supervisor should check for reflections on the monitor from windows and lights that could impair the student’s perception. To do this the supervisor should check by viewing the screen from the same position that the student will adopt.
If necessary, students should be shown how to indicate responses to the computer using the mouse, and when to respond (essentially when the tests will allow them to respond). This is particularly important when testing students with physical disabilities. As with any format assessment, students should not be allowed to take the tests if they are unwell, as results are likely to be unreliable.
Most students will experience no difficulties in understanding what is required of them when taking the tests in Exact, enabling them to follow the practice tasks easily and progress to the test phase without special attention from the teacher or supervisor. However, it is important that the administrator ensures that students understand the nature of the tasks in Exact: that they are tests and not games, and that they must work swiftly but thoughtfully and try their best at all times.
This is particularly important in the reading comprehension test, in which some students may be inclined automatically to click ‘Can’t tell’ if they are unable immediately to find the answer to a question within the passage. Students should be made aware that the answer can often be deduced or worked out from the information given in the passage, and that the ‘Can’t tell’ option should only be selected when there is insufficient information given in the passage to enable the answer to be deduced.
Students should also be made aware that although the reading comprehension test only lasts 10 minutes, they are not expected to finish it in that time and they should take enough time to get the earlier (easier) questions right.
In the rare event that a student does not understand the instructions spoken by the computer, the supervisor may re-express them in a more suitable manner. Explaining and re-expressing the task requirements to the student may continue into the demonstration and practice stages of each test. This is particularly useful for any student who is experiencing problems in understanding the true nature of the task. It is often easier for the student to comprehend the task requirements by experience of the practice stages than by more abstract oral explanation. Once the test items commence there should be no further aid given to the student.
It is usually not necessary for students to be closely supervised while attempting the tests, unless the teacher or administrator has a particular reason to do so, e.g. when assessing the need for exam access arrangements, where JCQ regulations require the assessor who signs the JCQ forms to carry out the assessment (see Section 1.2.6). As with all computer delivered group tests, Patoss recommends that when assessing for exam access purposes, Exact should be administrated carefully so that individual student responses can be observed and monitored. The tests in Exact have been designed to be interesting and stimulating for students in this age group and the vast majority of students are highly motivated to do their best. Once the teacher is satisfied that the student understands the requirements of a test, has completed the practice items and has moved on to the test items, the teacher may leave the student to complete that test.
Where the teacher suspects that a student may not be well motivated to complete the test, or may be easily distracted, or may be performing deliberately below their capabilities, closer supervision will be necessary. Disaffected students may display non-compliance by clicking on test items or answers at random, rather than thinking about the tasks and selecting answers after proper consideration. Such students, or those with very low ability, may need close supervision in order to provide encouragement and ensure they remain on task. This is particularly important in the reading comprehension test, which requires careful thought (see Sections 2.2 and 2.6.3).
In order for the assessment to be ‘fair’ (i.e. to give a reasonably accurate representation of the student’s abilities) it is essential to ensure that during the test:
- the student is paying attention, is ‘on task’, is not distracted and is trying their best
- the student does not become unduly fatigued
- there is no teaching or helping with the task during the test items (whether from the supervisor or other students)
- feedback from the supervisor is minimised and encouragement consistent (see further comments below).
Giving encouragement, prompts and feedback
As much as possible, the supervisor should avoid giving specific feedback to students during a test, because this may influence their behaviour in an undesirable fashion. This is good practice in any testing situation. There is a risk of feedback differentially affecting students, so that some are encouraged and others discouraged. Nevertheless, some students (particularly younger children or children with special educational needs) will try to elicit feedback from the supervisor about their performance. This may take the form of both verbal and nonverbal behaviours. For example, the student may ask directly if they were correct. Many students will look for the supervisor’s facial and bodily reactions to their responses. Some students may even try to evaluate the supervisor’s reaction by observing the supervisor’s reflection in the monitor screen. For these reasons it is usually preferable that if the supervisor is going to be near the student to observe the assessment, they should sit to the side and slightly behind the student to minimise any feedback to the student which may bias the results.
Rather than specific feedback, general encouragement should be given to the student. This encouragement should be referenced to task completion rather than task accuracy and ideally should be delivered equitably to all students. However, it is inevitable that some students will require more encouragement than others, and where this is the case the teacher should be mindful of the possibility of influencing results unduly. Differential encouragement between students is likely to have an influence on the results obtained and therefore should be avoided where possible. Some key phrases and general incentive prompts which may be used to aid the administration of the tests include: “well done”; “you were good at that; now try the next one”; “you will like this game”; “now concentrate on this”; “try hard”; “listen very carefully”; “have a go at these ones”; “have a try”; “just do your best”.