AUTOMATIC CLASSIFICATION OF EFL LEARNERS’ SELF-REPORTED TEXT DOCUMENTS ALONG AN AFFECTIVE CONTINUUM

Authors

DOI:

https://doi.org/10.20535/2410-8286.248091

Keywords:

affective factors, EFL learning, text classification, feature selection, EFL students, higher education, affective barriers

Abstract

This study aims to place EFL learners along an affective continuum via machine learning methods and present a new dataset about affective characteristics of EFL learners. In line with the purposes, written self-reports of 475 students from 5 different faculties in 3 universities in Turkey were collected and manually assigned by the researchers to one of the labels (positive, negative, or neutral). As a result, two combinations of the same dataset (AC-2 and AC-3) including different numbers of classes were used for the assessment of automatic classification approaches. Results revealed that automatic classification confirmed the manual classification to a great extent and machine learning methods could be used to classify EFL students along an affective continuum according to their affective characteristics. Maximum accuracy rate of automatic classification is 90.06% on AC-2 dataset including two classes. Similarly, on AC-3 dataset including three classes, maximum accuracy rate of classification is 71.79%. Last, the top-10 features/words obtained by feature selection methods are highly discriminative in terms of assessing student feelings for EFL learning. It could be stated that there is not an existing study in which feature selection methods and classifiers are used in the literature to automatically classify EFL learners’ feelings.

Downloads

Download data is not yet available.

Author Biographies

Derya Uysal, Alanya Alaaddin Keykubat University

Derya UYSAL is an Assistant Professor Doctor in School of Foreign Languages in Alanya Alaaddin Keykubat University in Antalya, Turkey.  She teaches EFL in preparatory program and English-medium departments of the university. She got a PhD degree in curriculum and instruction. Her research interests are EFL learning and teaching, affective domain, curriculum design, development and instruction.

Alper Kürşat Uysal, Alanya Alaaddin Keykubat University

Alper Kürşat UYSAL received the B.S. degree in computer engineering from Selcuk University, Turkey, in 2002, and the M.S. and Ph.D. degrees in computer science from Anadolu University, Turkey, in 2005 and 2013, respectively. He was a Visiting Research Fellow with the ECE Department, University of Michigan Dearborn, USA, from 2016 to 2017 for 12 months. He has been an Associate Professor Doctor in the Computer Engineering Department, Alanya Alaaddin Keykubat University, Turkey since 2021. His research interests include pattern recognition, text classification, and feature selection.

References

  1. Akın, A. A., & Akın, M. D. (2007). Zemberek, an open source nlp framework for Turkic languages. Structure, 10, 1-5. https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.556.69
  2. Anderson, L. W., & Bourke, S. F. (2000). Assessing affective characteristics in the schools. Routledge.
  3. Boudreau, C., MacIntyre, P., & Dewaele, J. M. (2018). Enjoyment and anxiety in second language communication: An idiodynamic approach. Studies in Second Language Learning and Teaching, 8(1), 149-170. https://doi.org/10.14746/ssllt.2018.8.1.7
    | |
  4. Brett, A., Smith, M., & Huitt, W. (2003). Overview of the affective domain. Educational Psychology Interactive. Valdosta, GA: Valdosta State University. http://www.edpsycinteractive.org/brilstar/chapters/affectdev.pdf
  5. Brown, H. D. (1973). Affective variables in second language acquisition. Language learning, 23(2), 231-244. https://doi.org/10.1111/j.1467-1770.1973.tb00658.x
    | |
  6. Buissink-Smith, N., Mann, S., & Shephard, K. (2011). How do we measure affective learning in higher education?. Journal of Education for Sustainable Development, 5(1), 101-114. https://doi.org/10.1177/097340821000500113
  7. Cahour, B. (2013). Characteristics, emergence and circulation in interactional learning. In M. Baker, J. Andriessen and S. Jarvela (eds), Affective learning together (pp.52-70). London: Routledge. https://doi.org/10.4324/9780203069684
  8. Chen, J., Huang, H., Tian, S., & Qu, Y. (2009). Feature selection for text classification with Naïve Bayes. Expert Systems with Applications, 36(3), 5432-5435. https://doi.org/10.1016/j.eswa.2008.06.054
  9. De Smet, A., Mettewie, L., Galand, B., Hiligsmann, P., & Van Mensel, L. (2018). Classroom anxiety and enjoyment in CLIL and non-CLIL: Does the target language matter?. Studies in Second Language Learning and Teaching, 8(1), 47-71. https://doi.org/10.14746/ssllt.2018.8.1.3
    | |
  10. Deng, X., Li, Y., Weng, J., & Zhang, J. (2019). Feature selection for text classification: A review. Multimedia Tools and Applications, 78(3), 3797-3816. https://doi.org/10.1007/s11042-018-6083-5
    |
  11. Dewaele, J. M., Magdalena, A. F., & Saito, K. (2019). The effect of perception of teacher characteristics on Spanish EFL learners’ anxiety and enjoyment. The Modern Language Journal, 103(2), 412-427. https://doi.org/10.1111/modl.12555
    |
  12. Dörnyei, Z. (1990). Conceptualizing motivation in foreign language learning. Language Learning, 40, 46-78. https://doi.org/10.1111/j.1467-1770.1990.tb00954.x
    | |
  13. Dörnyei, Z. (2005). The psychology of the language learner: Individual differences in second language acquisition. London: Erlbaum.
  14. Dörnyei, Z., Ryan, S. (2015). The psychology of the language learner revisited. New York, NY: Routledge.
  15. Dörnyei, Z., & Al-Hoorie, A. H. (2017). The motivational foundation of learning languages other than global English: Theoretical issues and research directions. Modern Language Journal, 101(3), 455-468. https://doi.org/10.1111/modl.12408
    | |
  16. Gardner, R.C. (1979). Social-psychological aspects of second language acquisition. In: Giles, H. & St. Clair, R. (eds.), Language and Social Psychology (pp.193–220). Oxford: Blackwell.
  17. Gardner, R. C. (1985). Social Psychology and Second Language Learning: The Role of Attitudes and Motivation. London: Edward Arnold.
  18. Gardner, R. C. (2001). Integrative motivation and second language acquisition. Motivation and second language acquisition, 23(1), 1-19.
  19. Gardner, R. C., & Clément, R. (1990). Social psychological perspectives on second language acquisition. John Wiley & Sons.
  20. Gardner, R.C. & Lambert, W.E. (1972). Motivational variables in second language acquisition. In R.C. Gardner & W. Lambert (eds.) Attitudes and motivation in second language learning. (pp. 119-216). Rowley, MA: Newbury House.
    |
  21. Garrett, P., & Young, R. F. (2009). Theorizing affect in foreign language learning: An analysis of one learner's responses to a communicative Portuguese course. The Modern Language Journal, 93(2), 209-226. https://doi.org/10.1111/j.1540-4781.2009.00857.x
    | |
  22. Griffith, K. G., & Nguyen, A. D. (2006). Are educators prepared to affect the affective domain. National forum of teacher education journal. 16 (3), 1-4.
  23. Horwitz, E. K. (1995). Student affective reactions and the teaching and learning of foreign languages. International Journal of Educational Research, 23(7), 573-579. https://doi.org/10.1016/0883-0355(96)80437-x
  24. Horwitz, E. K. (2000). It Ain't over'til It's Over: On Foreign Language Anxiety, First Language Deficits, and the Confounding of Variables. Modern Language Journal, 84(2), 256-259. https://doi.org/10.1111/0026-7902.00067
  25. Horwitz, E. K. (2001). Language anxiety and achievement. Annual review of applied linguistics, 21, 112-126. https://doi.org/10.1017/s0267190501000071
  26. Horwitz, E. K., Horwitz, M. B., ve Cope, J. (1986). Foreign language classroom anxiety. The modern language journal, 70(2), 125-132. https://doi.org/10.1111/j.1540-4781.1986.tb05256.x
    |
  27. Huensch, A., & Thompson, A. S. (2017). Contextualizing attitudes toward pronunciation: Foreign language learners in the United States. Foreign language annals, 50(2), 410-432. https://doi.org/10.1111/flan.12259
    |
  28. Jiang, L., Cai, Z., Zhang, H., & Wang, D. (2013). Naive Bayes text classifiers: a locally weighted learning approach. Journal of Experimental & Theoretical Artificial Intelligence, 25(2), 273-286. https://doi.org/10.1080/0952813x.2012.721010
  29. Joachims, T. (1998). Text categorization with support vector machines: Learning with many relevant features. In European conference on machine learning (pp. 137-142). Springer, Berlin, Heidelberg.
  30. Kębłowska, M. (2012). The place of affect in second language acquisition. In New perspectives on individual differences in language learning and teaching (pp. 157-167). Springer, Berlin, Heidelberg.
  31. Krashen, S. (1982). Principles and practice in second language acquisition. ISO 690
  32. Krathwohl, D. R., Bloom, B. S., and Masia, B. B. (1964). Taxonomy of educational objectives: The classification of educational goals. Handbook II: Affective domain, Allyn and Bacon, Boston, Mass.
  33. Laine, E. J. (1988). Report on The Affective Filter in Foreign Language Learning and Teaching. Report 2: A Validation Study of Filtering Factors with a Focus on the Learner's FL Self-Concept. Jyvaskyla Cross-Language Studies, No. 15.
  34. Liu, B., Xing, W., Zeng, Y., & Wu, Y. (2021). Quantifying the Influence of Achievement Emotions for Student Learning in MOOCs. Journal of Educational Computing Research, 59(3), 429-452. https://doi.org/10.1177/0735633120967318
    |
  35. McCoach, D. B., Gable, R. K., & Madura, J. P. (2013). Instrument development in the affective domain. New York, NY: Springer.
  36. Méndez López, M. G., & Peña Aguilar, A. (2013). Emotions as learning enhancers of foreign language learning motivation. Profile Issues in Teachers Professional Development, 15(1), 109-124. https://files.eric.ed.gov/fulltext/EJ1053728.pdf
  37. Nath, P. R., Mohamad, M., & Yamat, H. (2017). The effects of movies on the affective filter and English acquisition of low-achieving English learners. Creative Education, 8(08), 1357. https://doi.org/10.4236/ce.2017.88096
  38. Noels, K. A. (2005). Orientations to learning German: Heritage language background and motivational processes. Canadian Modern Language Review, 62, 285–312. https://doi.org/10.3138/cmlr.62.2.285
  39. Parlak, B., & Uysal, A. K. (2020). On classification of abstracts obtained from medical journals. Journal of Information Science, 46(5), 648-663. https://doi.org/10.1177/0165551519860982
  40. Pei, B., & Xing, W. (2021). An Interpretable Pipeline for Identifying At-Risk Students. Journal of Educational Computing Research, https://doi.org/10.1177/07356331211038168
    |
  41. Pierre, E., & Oughton, J. (2007). The Affective Domain: Undiscovered Country. College Quarterly, 10(4), 1-7. https://eric.ed.gov/?id=EJ813766
  42. Sampson, R. J. (2020). Interacting Levels and Timescales in the Emergence of Feelings in the L2 Classroom.In Richard J. Sampson and Richard S. Pinner (Eds.), Complexity Perspectives on Researching Language Learner and Teacher Psychology (pp.35-51). Bristol, Blue Ridge Summit: Multilingual Matters. https://doi.org/10.21832/9781788923569-004
  43. Schutz, P. A., & Lanehart, S. L. (2002). Emotions in education. Educational Psychologist, 37(2), 67-68. https://doi.org/10.1207/s15326985ep3702_1
  44. Setiawan, A., & Mardapi, D. (2019). The Development of Instrument for Assessing Students' Affective Domain Using Self-and Peer-Assessment Models. International Journal of Instruction, 12(3), 425-438. https://doi.org/10.29333/iji.2019.12326a
    |
  45. Shang, W., Huang, H., Zhu, H., Lin, Y., Qu, Y., & Wang, Z. (2007). A novel feature selection algorithm for text categorization. Expert Systems with Applications, 33(1), 1-5. https://doi.org/10.1016/j.eswa.2006.04.001
  46. Sousa, D. A (2016). How the Brain Works. Crowin Press.
  47. Uysal, A. K. (2016). An improved global feature selection scheme for text classification. Expert systems with Applications, 43, 82-92. https://doi.org/10.1016/j.eswa.2015.08.050
  48. Uysal, A. K., & Gunal, S. (2012). A novel probabilistic feature selection method for text classification. Knowledge-Based Systems, 36, 226-235. https://doi.org/10.1016/j.knosys.2012.06.005
  49. Wang, L. (2020). Application of Affective Filter Hypothesis in Junior English Vocabulary Teaching. Journal of Language Teaching and Research, 11(6), 983-987. https://doi.org/10.17507/jltr.1106.16
    |
  50. Zong, W., Wu, F., Chu, L. K., & Sculli, D. (2015). A discriminative and semantic feature selection method for text categorization. International Journal of Production Economics, 165, 215-222. https://doi.org/10.1016/j.ijpe.2014.12.035
  51. Zulfikar, T., Dahliana, S., & Sari, R. A. (2019). An Exploration of English Students’ Attitude toward Learning English. English Language Teaching Educational Journal, 2(1), 1-12. https://doi.org/10.12928/eltej.v2i1.947

Downloads

Published

2022-08-01

How to Cite

Uysal, D. ., & Uysal, A. K. (2022). AUTOMATIC CLASSIFICATION OF EFL LEARNERS’ SELF-REPORTED TEXT DOCUMENTS ALONG AN AFFECTIVE CONTINUUM. Advanced Education, 9(20), 4–14. https://doi.org/10.20535/2410-8286.248091

Issue

2022: Issue 20

Section

ARTICLES

License

Copyright (c) 2022 Derya Uysal, Alper Kürşat Uysal

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

    1. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  1. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).