The article deals with the innovative, cutting age solution within the language testing realm, namely computer adaptive language testing (CALT) in accordance with the NATO Standardization Agreement 6001 (NATO STANAG 6001) requirements for further implementation in foreign language training of personnel of the Armed Forces of Ukraine (AF of Ukraine) in order to increase the quality of foreign language testing. The research provides the CALT method developed according to NATO STANAG 6001 requirements and the CALT algorithm that contains three blocks: “Starting point”, “Item selection algorithm”, “Scoring algorithm” and “Termination criterion”. The CALT algorithm has an adaptive ability, changing a complexity level, sequence and the number of items according to the answers of a test taker. The comparative analysis of the results of the CALT method piloting and the paper-and-pencil testing (PPT) in reading and listening according to the NATO STANAG 6001 requirements justifies the effectiveness of the three-level CALT method. It allows us to determine the following important benefits of CALT: test length reduction, control of measurement accuracy, objective assessment, improved test security, generation of a unique set of items, adaptive ability of the CALT algorithm, high motivation of the test takers, immediate score reporting and test results management. CALT is a qualitative and effective tool to determine test takers’ foreign language proficiency level in accordance with NATO STANAG 6001 requirements within the NATO Defence Educational Enhancement Programme. CALT acquires a special value and relevance in the context of the global COVID 19 pandemic.


Albano, A. D., Cai, L., Lease, E. M., & McConnell, S. R. (2019). Computerized adaptive testing in early education: Exploring the impact of item position effects on ability estimation. Journal of Educational Measurement, 56(2), 437-451.

ATrainP-5. NATO STANAG 6001: Language Proficiency Levels. Edition A Version 2. (2016). North Atlantic Treaty Organization, Standardization Office. Retrieved 12 June 2017. Retrieved from

Babcock, B., & Weiss, D. J. (2012). Termination criteria in computerized adaptive tests: do variable-length CATs provide efficient and effective measurement? J. Comput. Adap. Test, 1, 1–18.

Bachman, L. (2000). Modern language testing at the turn of the century: assuring that what we count counts. Language Testing, 17, 1–42.

Bachman, L. F. (1990). Fundamental Considerations in Language Testing. Oxford, Oxford University Press. Retrieved from

Bachman, L. F., Davidson, F. & Milanovich, M. (1996). The use of test methods in the content analysis and design of EFL proficiency tests. Language Testing, 13, 125–1

Beckers, J.J., Schmidt, H.G. (2003) Computer experience and computer anxiety. Computers in Human Behavior, 19, 6, 785-797,

Blake, R. J. (2011). Current trends in online language learning. Annual Review of Applied Linguistics, 31, 19-35.

Canale, M., & Swain, M. (1980) Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47.

Chapelle, C., & Voss, E. (2017). Utilizing Technology in Language Assessment. Language Testing and Assessment. Encyclopedia of Language and Education, 3rd ed. (pp.149-161).

Chen, C. -., Wang, W. -., Chiu, M. M., & Ro, S. (2020). Item selection and exposure control methods for computerized adaptive testing with multidimensional ranking items. Journal of Educational Measurement, 57(2), 343-369.

Eggen, TJHM. (2018). Multi-Segment Computerized Adaptive Testing for Educational Testing Purposes. Front. Educ, 3, 111.

Fedoruk, P. I. (2008). Adaptive tests: general provisions. Mathematical machines and systems. 1, 115−127. Retrieved from

Frick, T. W. (1992). Computerized adaptive mastery tests as expert systems. Journal of Educational Computing Research, 8(2), 187−213.

Fulcher, G. (1999). Computerizing an English language placement test. ELT Journal, 53, 289–299.

Fulcher, G. (2017). Criteria for Evaluating Language Quality. Language Testing and Assessment. Encyclopedia of Language and Education, 3rd ed. (pp.179–192).

Gibbons, R. D., & deGruy, F. V. (2019). Without wasting a word: Extreme improvements in efficiency and accuracy using computerized adaptive testing for mental health disorders (CAT-MH). Current Psychiatry Reports, 21(8)

Hambleton, R. K., & Zaal, J. N. (1991). Advances in educational and psychological testing: theory and applications. Springer Sciences & Business Media, LLC.

Hambleton, R.K., Swaminathan, H. & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA, Sage Publications.

Hockly, N. (2019). Automated writing evaluation. ELT Journal, 73(1), 82–88.

Holzknecht, F., McCray, G., Eberharter, K., Kremmel, B., Zehentner, M., Spiby, R., & Dunlea, J. (2021). The effect of response order on candidate viewing behaviour and item difficulty in a multiple-choice listening test. Language Testing, 38(1), 41-61.

Hrabar, Е. V. (2010). Basic types of tests in foreign languages in US pedagogy. Scientific notes of Ternopil National Pedagogical University named after Volodymyr Hnatyuk. Series: Pedagogy, 1, 194–201. Retrieved from

ICAT (International Association for Computerized Adaptive Testing). Retrieved from

Jamieson, J. (2005). Trends in computer-based second language assessment. Annual Review of Applied Linguistics. 25. 228-242.

Kravchenko, O. M. & Plakasova, Zh.M. (2010). Model of intellectual controlling subsystem with multilevel adaptive testing. East European Journal of Advanced Technologies, 4/2 (46), 21–25.

Larkin, K. C. & Weiss, D. J. (1975). An empirical comparison of two-stage and pyramidal adaptive ability testing (Research Report, 75-1). Minneapolis: Psychometrics Methods Program, Department of Psychology, University of Minnesota. Retrieved from

Larson, J. (1999). Considerations for testing reading proficiency via computer-adaptive testing. In M. Chalhoub-Deville (Ed.), Studies in language testing, Vol. 10. Issues in computer-adaptive testing of reading proficiency (pp.71-90). Cambridge: University of Cambridge Press.

Lin, G.-H., Huang, Y.-J., Chou, Y.-T., Chiang, H.-Y., & Hsieh, C.-L. (2019). Computerized adaptive testing system of functional assessment of stroke. Journal of Visualized Experiments, 2019(143).

Lord, F. M. (1971). The self-scoring flexilevel test. Journal of Educational Measurement, 8, 147–151.

Maia, M., Lilley, M. & Barker, T. (2003). Computer-Adaptive Testing in Higher Education: the way forward? XXXVIII Cladea - Latin American Council of Schools of Administration: Lima.

Mâsse, L. C., O’Connor, T. M., Lin, Y., Hughes, S. O., Tugault-Lafleur, C. N., Baranowski, T., & Beauchamp, M. R. (2020). Calibration of the food parenting practice (FPP) item bank: Tools for improving the measurement of food parenting practices of parents of 5–12-year-old children. International Journal of Behavioral Nutrition and Physical Activity, 17(1).

Meunier, L. (1994). Computer Adaptive Language Tests (CALT) Offer a Great Potential for Functional Testing. Yet, Why Don't They? CALICO Journal, 11(4), 23-39. Retrieved February 25, 2021, from

Mizumoto, A., Sasao, Y., & Webb, S. A. (2019). Developing and evaluating a computerized adaptive testing version of the word part levels test. Language Testing, 36(1), 101-123.

Monfils, L. F., & Manna, V. F. (2021). Time to achieving a designated criterion score level: A survival analysis study of test taker performance on the TOEFL iBT® test. Language Testing, 38(1), 154-176. doi:10.1177/0265532220940709

Newhouse, C. P., & Cooper, M. (2013). Computer-based oral exams in Italian language studies. ReCALL, 25(3), 321-339.

Oppl, S., Reisinger, F., Eckmaier, A Helm, C. (2017). A flexible online platform for computerized adaptive testing. Int J Educ Technol High Educ, 14(2).

Padmavathi, M. (2016) A study of student-teachersʼ readiness to use computers in teaching: an empirical study. I-manager’s Journal on School Educational Technology, 11(3). Retrieved from

Sands, W.A., Waters, B.K. & McBride, J.R. (1997). Computerized adaptive testing. From inquiry to operation. Washington, American Psychological Association. Retrieved from

Serhiienko, V.P., Malezhyk, M.P., & Sitkar T.V. (2012). Computer technologies in testing: a textbook. Lutsk: Printing house “Volyn Polygraph”. Retrieved from

Thompson, N. A. & Weiss, D. A. (2011). A Framework for the Development of Computerized Adaptive Tests. Practical Assessment, Research, and Evaluation, 16, Article 1.

Thompson, N. A. (2007). A practitioner’s guide for variable-length computerized classification testing. Practical Assessment Research & Evaluation, 12 (1).

Thompson, N.A. (2016). User’s manual for SIFT: Software for investigating test fraud. Minneapolis, Assessment Systems Corporation.

Vispoel, W.P., Rocklin, T.R., & Wang, T. (1994). Individual differences and test administration procedures. A comparison of fixed-item, computerized adaptive, and self-adapted testing. Applied Measurement in Education, 7, 53–59.

Wang, C., Weiss, D. J., & Shang, Z. (2019). Variable-length stopping rules for multidimensional computerized adaptive testing. Psychometrika, 84(3), 749-771.

Weiss, D. J. & Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21, 361–375.

Weiss, D. J. (1973). The stratified adaptive computerized ability test (Research Report 73-3). Minneapolis: University of Minnesota, Department of Psychology. Retrieved from

Weiss, D. J. (2004). Computerized adaptive testing for effective and efficient measurement in counseling and education. Measurement and Evaluation in Counseling and Development, 37 (2), 70–84.

Wigglesworth, G. & Frost, K. (2017). Task and Performance-Based Assessment. Language Testing and Assessment. Encyclopedia of Language and Education, 3rd ed., 121-133.

Yigit, H. D., Sorrel, M. A., & de la Torre, J. (2019). Computerized adaptive testing for cognitively based multiple-choice data. Applied Psychological Measurement, 43(5), 388-401.




How to Cite

Gawliczek, P. ., Krykun, V., Tarasenko, N., Tyshchenko, M., & Shapran, O. (2021). COMPUTER ADAPTIVE LANGUAGE TESTING ACCORDING TO NATO STANAG 6001 REQUIREMENTS. Advanced Education, (17), 19–26.