FROM QUANTITY TO QUALITY: EVALUATING SCIENTIFIC WORK OF TERTIARY SCHOOL TEACHERS AND RESEARCHERS

(CASE STUDY OF EUROPEAN COUNTRIES AND UKRAINE)

Authors

DOI:

https://doi.org/10.20535/2410-8286.319629

Keywords:

Evaluation of Tertiary School Teachers, Evaluation of Scientific Activity, Open Science, Research Integrity

Abstract

The article is dedicated to analyzing modern approaches to evaluating the work of researchers in Europe, with a focus on qualitative indicators that serve as alternatives to traditional quantitative metrics. This research topic was selected in response to the growing number of discussions surrounding the challenges of evaluating scientific work, as well as the rise of initiatives being implemented by leading institutions and universities worldwide. The study examines the impact of initiatives such as the Leiden Manifesto, the San Francisco Declaration on Research Assessment (DORA) of 2012, the Hong Kong Principles, and the Open Science Career Evaluation Matrix (OS-CAM), as well as the European Research Area (ERA), on the development of new criteria for assessing scientific research. Special attention is given to the role of open science and inclusivity in evaluation processes. Additionally, examples of practices adopted by leading European universities and the potential for adapting these approaches in Ukraine are explored. The article emphasizes the importance and promise of qualitative evaluation of research activity, encouraging a shift away from conventional quantitative practices toward more meaningful and impactful assessment methods. The empirical method was employed to collect and analyze relevant information, forming the foundation of the research. An analytical method was also used to identify the strengths and weaknesses of the proposed evaluation practices. Furthermore, a comparative method was applied to investigate the differences between the suggested practices and to assess the effectiveness of each one. As a result of the study, global and national trends in the evaluation of researchers' work were identified, and recommendations were developed for implementing best practices of qualitative evaluation in Ukraine's academic sphere. The findings can be useful for Ukrainian higher education institutions when introducing internal evaluation practices for academic staff, as well as for central executive authorities responsible for shaping state education policy and initiating relevant legislative changes.

Downloads

Download data is not yet available.

Author Biographies

Iryna Izarova, Taras Shevchenko National University of Kyiv

Doctor of Law, Doctor of Science, Prof., Department of Justice

Oleksandr Bediukh, Taras Shevchenko National University of Kyiv

Ph.D., Senior Researcher, Research Department

Yuliia Hartman, Taras Shevchenko National University of Kyiv

PhD student, Department of Justice

Yuliia Baklazhenko, National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"

PhD (Pedagogics), Associate Professor, Department of Theory, Practice and Translation of the English Language

 

References

  1. Association of Universities in the Netherlands, Royal Netherlands Academy of Arts and Sciences, & Netherlands Organisation for Scientific Research. (2020). Strategy Evaluation Protocol, 2021–2027. https://www.universiteitenvannederland.nl/files/publications/SEP_2021-2027.pdf
  2. Bol, T., Vaan, M., & Rijt, A. (2018). The Matthew effect in science funding. PNAS, 115(19), 4887–4890. https://doi.org/10.1073/pnas.1719557115
    | |
  3. CoARA. (2022). Coalition for Advancing Research Assessment: The Agreement. https://coara.eu
  4. Decree of the Cabinet of Ministers of Ukraine "On Approval of the National Plan for Open Science" No. 892-p (2022, October 8). https://zakon.rada.gov.ua/laws/show/892-2022-p#Text
  5. Donahoe, C. (2024, June 18). DORA Initiatives Meeting: CLACSO-FOLEC on responsible assessment and open science. DORA. https://sfdora.org/2024/06/18/dora-initiatives-meeting-updates-from-clacso-folec/
  6. DORA. (2012). San Francisco Declaration on Research Assessment. https://sfdora.org/read/
  7. DORA. (2021). Tools to Advance Research Assessment (TARA) is a project to facilitate the development of new policies and practices for academic career assessment. https://sfdora.org/project-tara/
  8. DORA. (2022). Strategic Plan 2023–2026. https://sfdora.org/strategic-plan/
  9. European Molecular Biology Laboratory (EMBL). (2022). Research Assessment at EMBL. https://www.embl.org/about/research-assessment/
  10. European Research Council. (2023, April 28). Evaluation of ERC grant proposals: what to expect in 2024. https://erc.europa.eu/news-events/news/evaluation-erc-grant-proposals-what-expect-2024
  11. Felt, U., & Fochler, M. (2024). Quality assessment in berufungsprozessen an higher education institutions in Österreich: Eine analytische bestandsaufnahme von praktiken, perspektiven und herausforderungen. Institut für Wissenschafts- und Technikforschung, Universität Wien. https://phaidra.univie.ac.at/detail/o:2054391
  12. Garbuglia, F., Morais, R., Berghmans, S., & Gaillard, V. (2022). A closer look at research data practices in European universities: Follow-up to the 2020–21 EUA Open Science survey. European University Association. https://www.eua.eu/publications/reports/a-closer-look-at-research-data-practices-in-european-universities.html
  13. Ghent University. (2023, April 18). Career path and evaluation policy for Professorial Staff (ZAP). https://www.ugent.be/en/work/mobility-career/career-aspects/professorial-staff
  14. Global Young Academy, InterAcademy Partnership, & International Science Council. (2023). The future of research evaluation: A synthesis of current debates and developments. GYA-IAP-ISC Scoping Group. doi: 10.24948/2023.06. https://council.science/publications/the-future-of-research-evaluation-a-synthesis-of-current-debates-and-developments/
  15. Leiden Manifesto for Research Metrics. (2015). 10 principles to guide research evaluation with 25 translations, a video and a blog. http://www.leidenmanifesto.org/
  16. Maastricht University. (2020). Recognition & Rewards. https://www.maastrichtuniversity.nl/about-um/recognition-rewards
  17. Outhwaite, L. A., Gulliford, A., & Pitchford, N. J. (2019). A new methodological approach for evaluating the impact of educational intervention implementation on learning outcomes. International Journal of Research & Method in Education, 43(3), 225–242. https://doi.org/10.1080/1743727X.2019.1657081
    |
  18. Press Service of the Verkhovna Rada of Ukraine. (2024, June 7). The Verkhovna Rada of Ukraine adopted the Law "On Amendments to Certain Laws of Ukraine on Support of Scientific Work in Higher Education Institutions" (No. 9600). Verkhovna Rada of Ukraine. https://www.rada.gov.ua/news/razom/250490.html
  19. Prihna, T. (2020, June). About Ukrainian scientists and science. National Agency for Quality Assurance in Higher Education. https://naqa.gov.ua/2020/06/5687/
  20. Recognition & Rewards: Room for everyone's talent. (2019). https://recognitionrewards.nl
  21. Reed, M. S., Ferré, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M., & Holden, J. (2021). Evaluating impact from research: A methodological framework. Research Policy, 50(4), 104147. https://doi.org/10.1016/j.respol.2020.104147
  22. Resolution of the Cabinet of Ministers of Ukraine "On Approval of the Licensing Conditions for the Implementation of Educational Activities" No. 1187 (2015, December 30). https://zakon.rada.gov.ua/laws/show/1187-2015-p#Text
  23. Saenen, B., Morais, R., Gaillard, V., & Borrell-Damián, L. (2019). Research Assessment in the Transition to Open Science: 2019 EUA Open Science and Access Survey Results. European University Association. https://www.eua.eu/publications/reports/research-assessment-in-the-transition-to-open-science.html
  24. Science Europe. (2022). Research Assessment. https://scienceeurope.org/our-priorities/research-culture/research-assessment/
  25. Snackers, D. (2022). The Dutch initiative “Room for Everyone’s Tale”: Maastricht University’s approach. Personal in Hochschule und Wissenschaft entwickeln, (5), 49–63.
  26. Stec, P. (2022, Styczeń 21). System oceny pracy naukowej powinien być przejrzysty, stabilny i przewidywalny. Nauka w Polsce. https://naukawpolsce.pl/aktualnosci/news%2C91020%2Cprof-stec-system-oceny-pracy-naukowej-powinien-byc-przejrzysty-stabilny-i
  27. Steinhardt, G. (2020). Research Evaluation Practice in Austria. Informatics Europe. https://www.informatics-europe.org/images/national-associations/Research_Evaluation_in_Austria.pdf
  28. Strikha, M. (2017, November 14). University Science of Ukraine: On the Way from the USSR to Europe. Ministry of Education and Science of Ukraine. https://mon.gov.ua/news/universitetska-nauka-ukraini-na-shlyakhu-vid-srsr-do-evropi
  29. Universities Norway. (2021). NOR-CAM – A toolbox for recognition and rewards in academic careers. https://www.uhr.no/en/news-from-uhr/nor-cam-a-toolbox-for-recognition-and-rewards-in-academic-careers.5780.aspx
  30. University College London (UCL). (2018). Academic Career Framework and Promotions Processes: Information related to UCL’s promotion processes. https://www.ucl.ac.uk/human-resources/policies-advice/academic-career-framework-and-promotions-processes
  31. Utrecht University. (2021, February 11). Utrecht University presents new vision on Recognition and Rewards. https://www.uu.nl/en/news/utrecht-university-presents-new-vision-on-recognition-and-rewards
  32. O'Carroll, C., Rentier, B., Cabello Valdes, C., Esposito, F., Kaunismaa, E., Maas, K., Metcalfe, J., McAllister, D., & Vandevelde, K. (Eds.). (2017). Evaluation of research careers fully acknowledging Open Science practices: Rewards, incentives and/or recognition for researchers practicing Open Science. EU Publications Office. https://doi.org/10.2777/75255
  33. Węgrzyn, G. (2014). Problemy oceny pracowników naukowych. W S. Bilińskiego (Red.), Debaty PAU: T. 1. Oceny Nauki: Tomaszowice, 16–18 listopada 2013 (s. 55–64). Polska Akademia Umiejętności.
  34. World Conference on Research Integrity. (2019, June). Hong Kong Principles. https://www.wcrif.org/hong-kong-principles
  35. Wróblewska, M. N. (2022, April 7). Poland’s impact evaluation gets lost in translation. Research Professional News. https://www.researchprofessionalnews.com/rr-news-political-science-blog-2022-4-poland-s-impact-evaluation-gets-lost-in-translation/

Downloads

Published

2024-12-30

How to Cite

Izarova, I., Bediukh, O., Hartman, Y., & Baklazhenko, Y. (2024). FROM QUANTITY TO QUALITY: EVALUATING SCIENTIFIC WORK OF TERTIARY SCHOOL TEACHERS AND RESEARCHERS : (CASE STUDY OF EUROPEAN COUNTRIES AND UKRAINE) . Advanced Education, 12(25), 4–24. https://doi.org/10.20535/2410-8286.319629

Issue

Section

Editorial