What is PAT: Mathematics Adaptive?
PAT: Mathematics Adaptive is an online computer-adaptive version of a PAT: Mathematics test. In a PAT: Mathematics Adaptive test the computer selects the questions for each student based on the responses they have given to previous questions in the test. Students who struggle to answer the initial questions correctly are given easier questions until they begin to have some success. On the other hand, students who do answer the initial questions correctly are given progressively harder questions until they start to answer some of them incorrectly.
In effect, PAT: Mathematics Adaptive administers tests that are tailored to each individual student. Because they are well targeted in terms of difficulty level, an adaptive test will generally provide a more precise result for each student in a group than using one of the existing static PAT: Mathematics tests. In an adaptive test, the computer also makes sure that the questions being administered cover the normal range of mathematical topics assessed by PAT: Mathematics in the right proportions (number knowledge, number strategies, algebra, geometry and measurement, and statistics).
Who can complete an adaptive test?
The adaptive tests are suitable for students in Year 4 to Year 10. Students should be confident using a computer to complete a test. Many schools use adaptive testing for students who are working well above or well below their expected year level to gather depth in individual knowledge and skills. Other students in the same year level may sit the static test to ensure class-wide reporting.
How long is an adaptive test?
The adaptive test has from about 30 to 36 questions depending on the year level of the student. Students in the earlier year levels do shorter tests than students in later year levels. All students should be given 45 minutes to complete the test if you want to use the norm referenced scoring.
How difficult are the first questions in an adaptive test?
The adaptive tests select the first question on the basis of the student’s year level. For most students in each year level the first question would be considered as moderately easy. If a student gets the first question correct the computer administers a more difficult question next. Conversely, if the student gets the first question incorrect the computer administers an easier question next. The computer is programmed not to make the ‘jumps’ in difficulty too big.
How is achievement reported for PAT: Mathematics Adaptive?
PAT: Mathematics Adaptive tests report the same scale score and stanine information as the existing static PAT: Mathematics tests. At the end of an adaptive test the student’s achievement level is located on the PAT: Mathematics scale as a PATM scale score, and can be compared with the results of an appropriate national reference group using stanines.
Is the number of questions a student gets correct relevant in an adaptive test?
The number of questions a student has answered correctly in an adaptive test should not be used to interpret a result or make comparisons between students. The scale score is the best way to interpret how well a student has done. The adaptive tests are designed so that most students will answer about 60 percent of the questions they are administered correctly. Because the computer selects questions targeted at a students’ achievement level, some students will be administered harder questions than others, therefore the number correct is not relevant between students.
Some of NZCER Assist reports will show how many questions a student answered correctly and how many they were administered altogether. The number of questions they were administered is useful to know. Students should do between 30 and 36 questions in an adaptive test depending on their year level. When the number administered is much lower than this, it will usually mean that the student finished the test prematurely — for instance, by closing their browser. NZCER Assist will still report a scale score in this case, but the score will be based on an incomplete assessment, and the margin of error associated with the score will be large. If a student has not finished the test it will be displayed as incomplete. The test status can be found in List and Student reports when viewing the reports tab.
Which reports are available on NZCER Assist for an adaptive test?
NZCER Assist produces the same suite of reports for an assessment group that has completed an adaptive test as it does for a group that has completed one of the existing static tests, except the Item report. The Item reports are based on all students sitting the same question which is not available using the adaptive test. It is possible to see how each individual student did on the particular questions they were administered. This can be done by clicking on the question links provided in each student’s Individual Report.
How often should students complete an adaptive test?
The bank of PAT: Mathematics items available for adaptive testing is large enough so that students can complete about two adaptive tests per year without having to repeat a large number of the same questions each time.
Where do the PAT: Mathematics Adaptive questions come from?
PAT: Mathematics Adaptive draws on two sources of questions. The first is questions used in the existing static PAT: Mathematics tests. The second is a pool of new interactive items that have been developed and trialled by NZCER. The interactive items make use of a range of item types that are possible to administer on a computer, such as hot spot, drag and drop, and sorting items.
Which type of PAT: Mathematics test should I use?
The new adaptive tests and the existing static tests both have advantages. Which type you choose to use will depend on your reasons for testing and the kind of information you want to produce.
Adaptive test: tailors the content of the test to the individual student. This means that each student will be administered a test that is well-targeted to their achievement level. As well as ensuring that all students will have some success in terms of answering questions correctly, a well-targeted test also makes sure that each student’s achievement level is located on the PAT: Mathematics scale with a good level of precision. This is very difficult to achieve when we use the same static test with all students in a class. The variability between students in a typical New Zealand classroom is often quite large, meaning it is difficult to find one test form that will suit all students in the class.
Static tests: all students in a group sitting the same questions can sometimes be an advantage. When everyone in a group completes the same questions it becomes possible to look for patterns in students’ responses that may expose possible mathematical weaknesses (or strengths) across a class or year level cohort.
Regardless of which type of PAT Mathematics test you decide to use in your school, all results will be able to be reported as scale scores and stanines. Some schools may choose to use one type of test with some students e.g. well below and above students on adaptive - a quick way to know what they know and don't know and another type with others eg static test for most students to gain year level/gender/ethnicity strengths and needs - teacher capacity etc.
Key features of adaptive and static tests
Adaptive Test | Static Test |
Each student gets their own mix of questions targeted at their achievement level. There will be detailed individual level reporting but no group item level reporting available. | All students in the group respond to the same questions. This supports item level reporting. |
| The tests include a variety of question types — such as multiple choice, hot spot, drag and drop and sorting questions. | Only multiple choice questions are used. |
| Students need to answer each question before they can go on to the next question. | Students can omit a question and come back to it later. |
| Students cannot change an answer once they have answered a question and pressed ‘Next’. | Students can return to a question and change their answers. |