Education policy assessment of countries using an integrated decision-making approach

E. Ertugrul Karsak, Ece Ucar

Abstract

As governments strive to provide flawless and unbiased education to students while using resources in an efficient and sustainable manner, assessing countries' educational performance has become a prevailing topic, and the rising relevance of achieving this aim encourages the use of multi-criteria decision-making (MCDM) tools. This article presents an integrated approach using the Evaluation Based on Distance from Average Solution (EDAS) method in combination with the Best-Worst Method (BWM) to compare the educational performance in European Union member states. The performance assessment criteria are derived from the Programme for International Student Assessment (PISA) 2018 database and the UNESCO Institute for Statistics 2018 database. The research takes into account not only PISA test results but also further crucial features, including the teachers/students ratio, learning time, and government expenditure on primary education as a proportion of GDP, with the criteria weights computed using the linear version of BWM. The proposed approach, which uses EDAS in combination with BWM weights, produces a complete ranking of the evaluated EU nations, and also determines the best-performing country in terms of education. In order to illustrate the robustness of the proposed decision framework, a comparative analysis of the obtained rankings with the outcomes of other well-established distance-based MCDM methods is provided.

Keywords: educational performance of countries, performance assessment, MCDM, Best-Worst Method, EDAS

References

• Amavilah, V. H., & Rodríguez Andrés, A. (2022). Knowledge economy and the economic performance of African Countries: A seemingly unrelated and recursive approach. Journal of the Knowledge Economy, 15, 110-143. https://doi.org/10.1007/s13132-022-01033-7
• Aparicio, J., Cordero, J. M., Gonzalez, M., & Lopez-Espin, J. J. (2018). Using non-radial DEA to assess school efficiency in a cross-country perspective: An empirical analysis of OECD countries. Omega, 79, 9–20. https://doi.org/10.1016/j.omega.2017.07.004
• Aparicio, J., Perelman, S., & Santín, D. (2022). Comparing the evolution of productivity and performance gaps in education systems through DEA: an application to Latin American countries. Operational Research, 22, 1443-1477. https://doi.org/10.1007/s12351-020-00578-2
• Gebre, S. L., Cattrysse, D., & Van Orshoven, J. (2021). Multi-criteria decision-making methods to address water allocation problems: A systematic review. Water, 13(2), 125. https://doi.org/10.3390/w13020125
• Ginevicius, R. (2011). A new determining method for the criteria weights in multicriteria evaluation. International Journal of Information Technology & Decision Making, 10(6), 1067–1095. https://doi.org/10.1142/S0219622011004713
• Gupta, S., Verhoeven, M., & Tiongson, E. R. (2002). The effectiveness of government spending on education and health care in developing and transition economies. European Journal of Political Economy, 18(4), 717-737. https://doi.org/10.1016/S0176-2680(02)00116-7
• Hanushek, E. A., & Woessmann, L. (2020). A quantitative look at the economic impact of the European Union’s educational goals. Education Economics, 28(3), 225-244. https://doi.org/10.1080/09645292.2020.1719980
• Hwang, C. L., & Lin, M. J. (1987). Group decision making under multiple criteria: Methods and applications. Springer-Verlag.
• Hwang, C. L., & Yoon, K. (1981). Multiple attribute decision making. Methods and applications: a state-of-the-art survey. Springer-Verlag.
• Ishizaka, A., & Resce, G. (2021). Best-Worst PROMETHEE method for evaluating school performance in the OECD’s PISA project. Socio-Economic Planning Sciences, 73, 100799. https://doi.org/10.1016/j.seps.2020.100799
• Keshavarz-Ghorabaee, M., Zavadskas, E. K., Olfat, L., & Turskis, Z. (2015). Multi-criteria inventory classification using a new method of Evaluation Based on Distance from Average Solution (EDAS). Informatica, 26(3), 435–451. https://doi.org/10.15388/Informatica.2015.57
• Keshavarz-Ghorabaee, M., Zavadskas, E. K., Turskis, Z., & Antucheviciene, J. (2016). A new combinative distance-based assessment (CODAS) method for multi-criteria decision-making. Economic Computation & Economic Cybernetics Studies & Research, 50(3), 25-44.
• Keršuliene, V., Zavadskas, E. K., & Turskis, Z. (2010). Selection of rational dispute resolution method by applying new step‐wise weight assessment ratio analysis (SWARA). Journal of Business Economics and Management, 11(2), 243–258. https://doi.org/10.3846/jbem.2010.12
• Kocak, D., Ture, H., & Atan, M. (2019). Efficiency measurement with network DEA: an application to sustainable development goals 4. International Journal of Assessment Tools in Education, 6(3), 415-435. https://doi.org/10.21449/ijate.539487
• Lewis, S., & Lingard, B. (2022). Platforms, profits and PISA for schools: new actors, by-passes and topological spaces in global educational governance. Comparative Education, 59(1), 99-117. https://doi.org/10.1080/03050068.2022.2145006
• OECD. (2018). PISA 2018 Database. Retrieved June 4, 2023, from https://pisadataexplorer.oecd.org/ide/idepisa/dataset.aspx
• OECD. (2019). PISA 2018 Results (Volume I): What students know and can do. OECD Publishing. https://doi.org/10.1787/5f07c754-en
• Opricovic, S. (1998). Multicriteria optimization of civil engineering systems. Faculty of Civil Engineering, 2(1), 5-21.
• Opricovic, S., & Tzeng, G. H. (2004). Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research, 156(2), 445–455. https://doi.org/10.1016/S0377-2217(03)00020-1
• Pekelman, D., & Sen, S. K. (1974). Mathematical Programming Models for the Determination of Attribute Weights. Management Science, 20(8), 1217–1229. https://www.jstor.org/stable/2629998
• Psacharopoulos, G. (1994). Returns to investment in education: A global update. World Development, 22(9), 1325–1343. https://doi.org/10.1016/0305-750X(94)90007-8
• Ramsey, P. (1989). Critical Values for Spearman’s Rank Order Correlation. Journal of Educational Statistics, 14(3), 245–253. https://doi.org/10.3102/10769986014003245
• Rezaei, J. (2015). Best-worst multi-criteria decision-making method. Omega, 53, 49–57. https://doi.org/10.1016/j.omega.2014.11.009
• Rezaei, J. (2016). Best-worst multi-criteria decision-making method: Some properties and a linear model. Omega, 64, 126–130. https://doi.org/10.1016/j.omega.2015.12.001
• Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234–281. https://doi.org/10.1016/0022-2496(77)90033-5
• Saaty, T. L. (2004). Decision making – the Analytic Hierarchy and Network Processes (AHP/ANP). Journal of Systems Science and Systems Engineering, 13(1), 1–35. https://doi.org/10.1007/s11518-006-0151-5
• Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
• Srinivasan, V., & Shocker, A. D. (1973). Linear programming techniques for multidimensional analysis of preferences. Pyschometrika, 38(3), 337-369. https://doi.org/10.1007/BF02291658
• Stamenković, M., Anić, I., Petrović, M., & Bojković, N. (2016). An ELECTRE approach for evaluating secondary education profiles: evidence from PISA survey in Serbia. Annals of Operations Research, 245(1–2), 337–358. https://doi.org/10.1007/s10479-015-1823-7
• Ucar, E., & Karsak, E. E. (2021). Educational Performance Assessment of OECD Countries Using PISA 2018 Data. Proceedings of IAC 2021 in Vienna (pp. 64-73). https://www.conferences-scientific.cz/file/9788088203223
• UNESCO. (2018). UNESCO Institute for Statistics 2018 Database. Retrieved June 4, 2023, from http://data.uis.unesco.org
• Yu, P. L. (1973). A class of solutions for group decision problems. Management Science 19(8), 936–946. https://doi.org/10.1287/mnsc.19.8.936
• Zeleny, M. (1982). Multiple criteria decision making. Mc-Graw-Hill.