FROM QUANTITY TO QUALITY: EVALUATING SCIENTIFIC WORK OF TERTIARY SCHOOL TEACHERS AND RESEARCHERS
(CASE STUDY OF EUROPEAN COUNTRIES AND UKRAINE)
DOI:
https://doi.org/10.20535/2410-8286.319629Keywords:
Evaluation of Tertiary School Teachers, Evaluation of Scientific Activity, Open Science, Research IntegrityAbstract
The article is dedicated to analyzing modern approaches to evaluating the work of researchers in Europe, with a focus on qualitative indicators that serve as alternatives to traditional quantitative metrics. This research topic was selected in response to the growing number of discussions surrounding the challenges of evaluating scientific work, as well as the rise of initiatives being implemented by leading institutions and universities worldwide. The study examines the impact of initiatives such as the Leiden Manifesto, the San Francisco Declaration on Research Assessment (DORA) of 2012, the Hong Kong Principles, and the Open Science Career Evaluation Matrix (OS-CAM), as well as the European Research Area (ERA), on the development of new criteria for assessing scientific research. Special attention is given to the role of open science and inclusivity in evaluation processes. Additionally, examples of practices adopted by leading European universities and the potential for adapting these approaches in Ukraine are explored. The article emphasizes the importance and promise of qualitative evaluation of research activity, encouraging a shift away from conventional quantitative practices toward more meaningful and impactful assessment methods. The empirical method was employed to collect and analyze relevant information, forming the foundation of the research. An analytical method was also used to identify the strengths and weaknesses of the proposed evaluation practices. Furthermore, a comparative method was applied to investigate the differences between the suggested practices and to assess the effectiveness of each one. As a result of the study, global and national trends in the evaluation of researchers' work were identified, and recommendations were developed for implementing best practices of qualitative evaluation in Ukraine's academic sphere. The findings can be useful for Ukrainian higher education institutions when introducing internal evaluation practices for academic staff, as well as for central executive authorities responsible for shaping state education policy and initiating relevant legislative changes.
Downloads
References
- Association of Universities in the Netherlands, Royal Netherlands Academy of Arts and Sciences, & Netherlands Organisation for Scientific Research. (2020). Strategy Evaluation Protocol, 2021–2027. https://www.universiteitenvannederland.nl/files/publications/SEP_2021-2027.pdf
- Bol, T., Vaan, M., & Rijt, A. (2018). The Matthew effect in science funding. PNAS, 115(19), 4887–4890. https://doi.org/10.1073/pnas.1719557115
| | - CoARA. (2022). Coalition for Advancing Research Assessment: The Agreement. https://coara.eu
- Decree of the Cabinet of Ministers of Ukraine "On Approval of the National Plan for Open Science" No. 892-p (2022, October 8). https://zakon.rada.gov.ua/laws/show/892-2022-p#Text
- Donahoe, C. (2024, June 18). DORA Initiatives Meeting: CLACSO-FOLEC on responsible assessment and open science. DORA. https://sfdora.org/2024/06/18/dora-initiatives-meeting-updates-from-clacso-folec/
- DORA. (2012). San Francisco Declaration on Research Assessment. https://sfdora.org/read/
- DORA. (2021). Tools to Advance Research Assessment (TARA) is a project to facilitate the development of new policies and practices for academic career assessment. https://sfdora.org/project-tara/
- DORA. (2022). Strategic Plan 2023–2026. https://sfdora.org/strategic-plan/
- European Molecular Biology Laboratory (EMBL). (2022). Research Assessment at EMBL. https://www.embl.org/about/research-assessment/
- European Research Council. (2023, April 28). Evaluation of ERC grant proposals: what to expect in 2024. https://erc.europa.eu/news-events/news/evaluation-erc-grant-proposals-what-expect-2024
- Felt, U., & Fochler, M. (2024). Quality assessment in berufungsprozessen an higher education institutions in Österreich: Eine analytische bestandsaufnahme von praktiken, perspektiven und herausforderungen. Institut für Wissenschafts- und Technikforschung, Universität Wien. https://phaidra.univie.ac.at/detail/o:2054391
- Garbuglia, F., Morais, R., Berghmans, S., & Gaillard, V. (2022). A closer look at research data practices in European universities: Follow-up to the 2020–21 EUA Open Science survey. European University Association. https://www.eua.eu/publications/reports/a-closer-look-at-research-data-practices-in-european-universities.html
- Ghent University. (2023, April 18). Career path and evaluation policy for Professorial Staff (ZAP). https://www.ugent.be/en/work/mobility-career/career-aspects/professorial-staff
- Global Young Academy, InterAcademy Partnership, & International Science Council. (2023). The future of research evaluation: A synthesis of current debates and developments. GYA-IAP-ISC Scoping Group. doi: 10.24948/2023.06. https://council.science/publications/the-future-of-research-evaluation-a-synthesis-of-current-debates-and-developments/
- Leiden Manifesto for Research Metrics. (2015). 10 principles to guide research evaluation with 25 translations, a video and a blog. http://www.leidenmanifesto.org/
- Maastricht University. (2020). Recognition & Rewards. https://www.maastrichtuniversity.nl/about-um/recognition-rewards
- Outhwaite, L. A., Gulliford, A., & Pitchford, N. J. (2019). A new methodological approach for evaluating the impact of educational intervention implementation on learning outcomes. International Journal of Research & Method in Education, 43(3), 225–242. https://doi.org/10.1080/1743727X.2019.1657081
| - Press Service of the Verkhovna Rada of Ukraine. (2024, June 7). The Verkhovna Rada of Ukraine adopted the Law "On Amendments to Certain Laws of Ukraine on Support of Scientific Work in Higher Education Institutions" (No. 9600). Verkhovna Rada of Ukraine. https://www.rada.gov.ua/news/razom/250490.html
- Prihna, T. (2020, June). About Ukrainian scientists and science. National Agency for Quality Assurance in Higher Education. https://naqa.gov.ua/2020/06/5687/
- Recognition & Rewards: Room for everyone's talent. (2019). https://recognitionrewards.nl
- Reed, M. S., Ferré, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M., & Holden, J. (2021). Evaluating impact from research: A methodological framework. Research Policy, 50(4), 104147. https://doi.org/10.1016/j.respol.2020.104147
- Resolution of the Cabinet of Ministers of Ukraine "On Approval of the Licensing Conditions for the Implementation of Educational Activities" No. 1187 (2015, December 30). https://zakon.rada.gov.ua/laws/show/1187-2015-p#Text
- Saenen, B., Morais, R., Gaillard, V., & Borrell-Damián, L. (2019). Research Assessment in the Transition to Open Science: 2019 EUA Open Science and Access Survey Results. European University Association. https://www.eua.eu/publications/reports/research-assessment-in-the-transition-to-open-science.html
- Science Europe. (2022). Research Assessment. https://scienceeurope.org/our-priorities/research-culture/research-assessment/
- Snackers, D. (2022). The Dutch initiative “Room for Everyone’s Tale”: Maastricht University’s approach. Personal in Hochschule und Wissenschaft entwickeln, (5), 49–63.
- Stec, P. (2022, Styczeń 21). System oceny pracy naukowej powinien być przejrzysty, stabilny i przewidywalny. Nauka w Polsce. https://naukawpolsce.pl/aktualnosci/news%2C91020%2Cprof-stec-system-oceny-pracy-naukowej-powinien-byc-przejrzysty-stabilny-i
- Steinhardt, G. (2020). Research Evaluation Practice in Austria. Informatics Europe. https://www.informatics-europe.org/images/national-associations/Research_Evaluation_in_Austria.pdf
- Strikha, M. (2017, November 14). University Science of Ukraine: On the Way from the USSR to Europe. Ministry of Education and Science of Ukraine. https://mon.gov.ua/news/universitetska-nauka-ukraini-na-shlyakhu-vid-srsr-do-evropi
- Universities Norway. (2021). NOR-CAM – A toolbox for recognition and rewards in academic careers. https://www.uhr.no/en/news-from-uhr/nor-cam-a-toolbox-for-recognition-and-rewards-in-academic-careers.5780.aspx
- University College London (UCL). (2018). Academic Career Framework and Promotions Processes: Information related to UCL’s promotion processes. https://www.ucl.ac.uk/human-resources/policies-advice/academic-career-framework-and-promotions-processes
- Utrecht University. (2021, February 11). Utrecht University presents new vision on Recognition and Rewards. https://www.uu.nl/en/news/utrecht-university-presents-new-vision-on-recognition-and-rewards
- O'Carroll, C., Rentier, B., Cabello Valdes, C., Esposito, F., Kaunismaa, E., Maas, K., Metcalfe, J., McAllister, D., & Vandevelde, K. (Eds.). (2017). Evaluation of research careers fully acknowledging Open Science practices: Rewards, incentives and/or recognition for researchers practicing Open Science. EU Publications Office. https://doi.org/10.2777/75255
- Węgrzyn, G. (2014). Problemy oceny pracowników naukowych. W S. Bilińskiego (Red.), Debaty PAU: T. 1. Oceny Nauki: Tomaszowice, 16–18 listopada 2013 (s. 55–64). Polska Akademia Umiejętności.
- World Conference on Research Integrity. (2019, June). Hong Kong Principles. https://www.wcrif.org/hong-kong-principles
- Wróblewska, M. N. (2022, April 7). Poland’s impact evaluation gets lost in translation. Research Professional News. https://www.researchprofessionalnews.com/rr-news-political-science-blog-2022-4-poland-s-impact-evaluation-gets-lost-in-translation/
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Iryna Izarova, Oleksandr Bediukh, Yuliia Hartman, Yuliia Baklazhenko. The cover for this article was designed using pixlr.com
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).