Frequently Asked Questions about Computer-Adaptive Testing (CAT)

What is a Computer-Adaptive Test?

As test takers respond to test items, a computer-adaptive test "adapts" itself to test takers by selecting the next item to be presented on the basis of performance on preceding items.

For example, if a test taker performs well on a set of intermediate-level questions, the computer will next present a set of questions at the advanced level. If the test taker performs poorly on the intermediate-level questions, the computer will select novice-level items as the next question set. Thus, the CAT is constantly attempting to establish the appropriate level for the test taker's performance, and stops testing once the performance at a given level is shown to be the test taker's highest sustainable performance.

Why is there a need for computer-adaptive tests in language teaching?

As interest grows in performance-based testing, which requires students to demonstrate an ability to use what they have learned, new ways to assess performance must be developed. Since 1986, the University of Minnesota's College of Liberal Arts has had a foreign language requirement based on demonstrated proficiency rather than on the number of language courses completed.

To date, the University uses computer-based tests and oral interviews to evaluate students' target language proficiency in reading, writing, listening, and speaking. The requirement that students demonstrate proficiency has had significant positive impacts on second language curriculum design, teaching, and evaluation across the state of Minnesota. The use of computer-based technology greatly facilitates test administration, scoring, data analysis and data management, and score reporting. Thus, computer-based testing enhances the feasibility of replicating Minnesota's proficiency-based model at other institutions. Although CARLA's current computer-based tests are not yet fully adaptive, their design reflects all the advantages of CAT listed below.

What are the advantages of CAT?

In addition to the issues of cost and efficiency addressed above, advantages of CAT include:

• Compared to paper-and-pencil tests, CAT technology requires fewer test items to arrive at a more accurate estimate of test takers' language proficiency.
• CAT scoring allows for finer distinctions than total number correct.
• CAT scoring takes into account not just the number of item answered correctly, but which items were answered correctly. A test taker who correctly answers a more difficult set of questions will score higher than a test taker who correctly answers an easier set of questions.
• The time required to take a CAT is shorter, since test items outside the test taker's proficiency level are excluded.
• The test taker is continuously faced with a realistic challenge--items are not too difficult or too easy.
• Because each test taker is potentially administered a different set of test items, test security is enhanced.
• CAT technology allows test takers to receive immediate feedback on their performance.
• For tests administered on a large scale, scheduling and supervision concerns are greatly reduced because individual administration is possible.
• CAT technologies have been found to improve test-taking motivation and to reduce average test score differences across ethnic groups (Pine, Church, Gialluca, and Weiss, 1979; Pine and Weiss, 1978).
• Students' performance over time can be tracked by using the computer to store performance data.