The use of new statistical procedures such as Item Response Theory has greatly facilitated the development of computer-adaptive tests, where the adaptiveness is based on measures of item difficulty resulting from the performance of trial candidates. However, studies into the acquisition of L2 grammar by learners with different Lls indicate that the learners' L1 strongly influences their acquisition of grammar in the L2 [Lado, R. (1957) Linguistics Across Cultures: Applied Linguistics for Language Teachers; Rutherford, W. and Sharwood-Smith, M. (1985) Applied Linguistics, 6(3), 274–282; Zobl, H. (1980) Language Learning, 30, 33–57; Zobl, H. (1982) TESOL Quarterly, 16(2), 169–183]. Thus, it would be expected that grammar test items would present different levels of difficulty to candidates from different language backgrounds. Where a computer-adaptive grammar test is to be used with such candidates it is, therefore, questionable whether set item difficulty measures can validly be used for all types of candidate. The study investigates the performance of students from different language backgrounds, using data from a computer-adaptive Japanese grammar test developed as a placement tool. The trial pen-and-paper test consisted of 225 multiple choice items. Fourteen hundred students in Australia, China and Japan (all of whom had studied Japanese for 150–500 hrs) each completed 50 items. In this study, data is presented from native speakers of English and Chinese. Item difficulties were found to be quite different for the three groups of candidates. This has implications for the validity of use of computer-adaptive tests, in that where actual candidates are from a different background from that of the trial population not only does the test fail to measure such candidates efficiently, but their measures of ability are also affected.