Application of the AHP method to create a ranking of criteria for assessing the reliability of websites

Adam Czerwiński


Knowledge of how to obtain, evaluate, and use information effectively is essential not only for students and teachers (including the academic ones), but also for website designers, creators and content providers. This particularly applies to the possibility of assessing the credibility of websites. The problem is not only to select the appropriate categories and criteria for this assessment, but also to include all these criteria in the final assessment.

The aim of the article is to establish a ranking of the validity of the criteria for assessing the reliability of websites from the point of view of experts. This required answers to the following questions:

  • How to create a framework for assessing the credibility of websites?
  • Which of the multi-criteria assessment methods should be chosen to assess the validity of the criteria?
  • Which of the criteria for assessing the credibility of websites should be considered as the most important?

Saaty's AHP method was used to assess the validity of the criteria. The result of these studies is a ranking of importance for a set of selected 21 criteria for assessing the reliability of sites. It turned out that, according to experts, the quality of information and authenticity of the operator were at the top of the ranking. However, criteria such as functionality, performance and clarity of the sites and their appearance and structure are of secondary importance. A modern website must represent a sufficiently high level of its construction and operate within an appropriate infrastructure, so it should be reliable, functional, efficient and transparent. Therefore, the criteria listed in the expert opinion are now becoming less important.

Keywords: credibility, website, evaluation criteria, importance, ranking


  • Aggarwal, S., Van Oostendorp, H., Reddy, Y. R. i Indurkhya, B. (2014). Providing web credibility assessment support. W Proceedings of the 2014 European Conference on Cognitive Ergonomics (s. 1-8). ACM.
  • Banayoun, R., Roy, B., Sussman, N. (1966). Manual de référence du programme Electre. Note de Synthése et Formation, 25. Direction Scientifique SEMA.
  • Brans, J. P. (1982). L'ingenierie de la decision. Elaboration d'instruments d'aide a la decision. La methode PROMETHEE. W R. Nadeau i M. Landry (red.), L'aide a la decision: nature, instruments et perspectives d'avenir. Presses de l'Universite Laval.
  • Brans, J. P. i Vincke, Ph. (1985). A preference ranking organization method: (The PROMETHEÉ method for multiple criteria decision-making). Management Science, 31(6), 647-656.
  • Brans, J. P. i Mareschal, B. (2005). PROMETHEE methods. W J. Figueira, S. Greco i M. Ehrgott (red.), Multiple criteria decision analysis: state of the art surveys (s. 163-195). Springer.
  • Choi, W. i Stvilia, B. (2015). Web credibility assessment: conceptualization, operationalization, variability, and models. Journal of the Association for Information Science and Technology, 66(12), 2399-2414.
  • Czerwiński, A. (2016). Ocena wiarygodności serwisów www polskich uniwersytetów. Praktyka i Teoria Informacji Naukowej i Technicznej, 24(2-3), 44-55.
  • Czerwiński, A. (2019a). Identyfikacja i porównanie kryteriów oceny wiarygodności witryn internetowych. Przegląd Organizacji, 8, 50-58.
  • Czerwiński, A. (2019b). Ważność kryteriów oceny wiarygodności witryn internetowych na podstawie badań. e-mentor, 4(81), 39-46.
  • Fogg, B. (2003). Persuasive technology: Using computers to change what we think and do. Morgan Kaufmann.
  • Fogg, B., Soohoo, C., Danielson, D. R., Marable, L., Stanford, J. i Tauber, E. R. (2003). How do users evaluate the credibility of Web sites?: a study with over 2,500 participants. W Proceedings of the 2003 conference on Designing for user experiences (s. 1-15). ACM.
  • Goepel, K. D. (2013). Implementing the analytic hierarchy process as a standard method for multi-criteria decision making in corporate enterprises - a new AHP Excel template with multiple inputs. W Proceedings of the International Symposium on the Analytic Hierarchy Process 2013 (s. 1-10).
  • Golbeck, J. (2006). Trust on the World Wide Web: A survey. Foundation and Trends in Web Science, 1(2), 131-197.
  • Hinloopen, E., Nijkamp, P. i Rietveld, P. (1983). The REGIME method: a new multicriteria technique. W P. Hansen (red.), Essays and surveys on multiple criteria decision making (s. 146-155). Springer.
  • Hwang, C. L. i Yoon, K. (1981). Multiple attribute decision making. Methods and applications. Springer.
  • Jacquet-Lagréze, E. i Siskos, J. (1982). Assessing a set of additive utility functions for multicriteria decision making: the UTA method. European Journal of Operational Research, 10(2), 151-164.
  • Kąkol, M. i Nielek, R. (2015). What affects web credibility perception? An analysis of textual justifications. Computer Science, 16(3), 295-310.
  • Keeney, R. L. i Raiffa, H. (1993). Decisions with multiple objectives-preferences and value tradeoffs. Cambridge University Press.
  • Malczewski, J. (1999). GIS and multicriteria decision analysis. JohnWiley & Sons.
  • Metzger, M. J. (2007). Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology, 58(13), 2078-2091.
  • Metzger, M. J., Flanagin, A. J., Eyal, K., Lemus, D. R. i McCann, R. M. (2003). Credibility for the 21st century: Integrating perspectives on source, message, and media credibility in the contemporary media environment. Annals of the International Communication Associaton, 27, 293-336.
  • Oltenau, A., Peshterliev, S., Liu, X. i Aberer, K. (2013). Web credibility: features exploration and credibility prediction. W P. Serdyukov, P. Braslavski, S. O. Kuznetsov, J. Kamps, S. Rüger, E. Agichtein, I. Segalovich i E. Yilmaz (red.), Advances in information retrieval (s. 557-568). Springer.
  • Piwowarski, M. (2009). Wielokryterialna analiza decyzyjna w systemach GIS. Studia i Materiały Polskiego Stowarzyszenia Zarządzania Wiedzą, 18, 123-134.
  • Rieh, S. Y. i Danielson, D. R. (2007). Credibility: A multidisciplinary framework. Annual Review of Information Science and Technology, 41(1), 307-364.
  • Roubens, M. (1982). Preference relation in actions and criteria in multicriteria decision making. European Journal of Operational Research, 10(1), 51-55.
  • Roy, B. (1990). Wielokryterialne wspomaganie decyzji. WNT.
  • Roy, B. i Bouyssou, D. (1993). Aide multicritere a la decision: methodes at cas. Economica.
  • Saaty, T. L. (1980). Multicriteria decision making: the analytic hierarchy process. McGraw-Hill.
  • Saaty, T. L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48(1), 9-26.
  • Saaty, T. L. (1994a). How to make a decision: the analytic hierarchy process. INFORMS Journal on Applied Analytics, 24(6), 19-43.
  • Saaty, T. L. (1994b). Mathematical principles of decisions making: the complete theory of the analytical hierarchy process. RWS Publications.
  • Saaty, T. L. (2004). Decision making - the Analytic Hierarchy and Network Processes (AHP/ANP). Journal of Systems Science and Systems Engineering, 13(1), 1-34.
  • Trzaskalik, T. (red.). (2014). Wielokryterialne wspomaganie decyzji. Metody i zastosowanie. Polskie Wydawnictwo Ekonomiczne.
  • Vansnick, J. C. (1986). On the problem of weights in multiple criteria decision making (the noncompensatory approach). European Journal of Operational Research, 24(2), 288-294.
  • Von Winterfeldt, D. i Edwards, W. (1986). Decision analysis and behavioural research. Cambridge University Press.

Adam Czerwiński

About the article


The article is in the printed version on pages 64-75.

pdf read the article (Polish)

How to cite

Czerwiński, A. (2020). Zastosowanie metody AHP do tworzenia rankingu kryteriów oceny wiarygodności witryn internetowych. e-mentor, 5(87), 64-75.