Chapter 24: Introducing the systematic science mapping framework: an innovative and mixed approach for macro scale reviews
Open access

Systematic Science Mapping (SSM) is a novel mixed methods research (MMR) design for literature reviews of large scale, thousands of publications, including entire scientific fields. SSM establishes a “big picture” view of a field’s evolution, a thematic analysis of the research in a field, and synthesizes findings even in the presence of conceptual overlaps or inconsistencies. An overview of its roots in systematic literature reviews (SLRs) and science mapping is presented first before integrating them in a sequential mixed models design. Then, the application of SSM is illustrated in the field of responsible artificial intelligence (RAI). Evolutionary maps are presented as a tool for visualising the semantic drift of ethical principles over time. Based on “thick data”, SSM shows a way of emphasising commonalities over differences for reducing the academic-to-practice gap in RAI. Guiding notes are provided to those who may wish to employ this MMR design.

  • Adams, R., Smart, P., & Huff, A. (2017). Shades of grey. International Journal of Management Reviews, 19(4), 432–454. https://doi.org/10.1111/ijmr.12102

  • Aromataris, E., & Pearson, A. (2014). The systematic review. The American Journal of Nursing, 114(3), 53–58.

  • Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018). The Moral Machine experiment. Nature, 563(7729), 59–64. https://doi.org/10.1038/s41586-018-0637-6

  • Barczak, G. (2017). Writing a review article. Journal of Product Innovation Management, 34(2), 120–121.

  • Best, M., Knight, P., Lietz, P., Lockwood, C., Nugroho, D., & Tobin, M. (2013). The impact of national and international assessment programmes on education policy, particularly policies regarding resource allocation and teaching and learning practices in developing countries. EPPI-Centre. https://research.acer.edu.au/cgi/viewcontent.cgi?article=1016&context=ar_misc

  • Björneborn, L., & Ingwersen, P. (2004). Toward a basic framework for webometrics. Journal of the American Society for Information Science and Technology, 55(14), 1216–1227.

  • Blümel, C., & Schniedermann, A. (2020). Studying review articles in scientometrics and beyond: A research agenda. Scientometrics, 124(1), 711–728. https://doi.org/10.1007/s11192-020-03431-7

  • Bornmann, L., & Haunschild, R. (2017). Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact? Scientometrics, 110(2), 937–943.

  • Briner, R., & Denyer, D. (2012). Systematic review and evidence synthesis as a practice and scholarship tool. In M. Rousseau (Ed.), Handbook of Evidence-based Management: Companies, Classrooms and Research (pp. 112–129). Oxford University Press.

  • Cameron, R., & Herrmann, H. (2023). Ethical issues for mixed methods research in an era of unprecedented digital transformations. In C. Poth (Ed.), The Handbook of Mixed Methods Research Designs (pp. 154–165). Sage.

  • Cameron, R., Herrmann, H., & Reynolds, G. (2023). Historical and comparative perspectives on MMR in business, the social sciences and medicine. In R. Cameron & X. Golenko (Eds), Handbook of Mixed Methods Research in Business and Management (pp. 11–26). Edward Elgar Publishing.

  • Charmaz, K. (2014). Constructing Grounded Theory. Sage.

  • Chellappandi, P., & Vijayakumar, C. (2018). Bibliometrics, scientometrics, webometrics/cybermetrics, informetrics and altmetrics – An emerging field in library and information science research. Shanlax International Journal of Education, 7(1), 5–8.

  • Chen, C. (2017). Science mapping: A systematic review of the literature. Journal of Data and Information Science, 2(2), 1–40.

  • Choudhri, A., Siddiqui, A., Khan, N., & Cohen, H. (2015). Understanding bibliometric parameters and analysis. Radiographics, 35(3), 736–746. https://doi.org/10.1148/rg.2015140036

  • Cobo, M., López-Herrera, A., Herrera-Viedma, E., & Herrera, F. (2011a). An approach for detecting, quantifying, and visualizing the evolution of a research field. Journal of Informetrics, 5(1), 146–166.

  • Cobo, M., López-Herrera, A., Herrera-Viedma, E., & Herrera, F. (2011b). Science mapping software tools. Journal of the American Society for Information Science and Technology, 62(7), 1382–1402. https://doi.org/10.1002/asi.21525

  • Cobo, M., López-Herrera, A., Herrera-Viedma, E., & Herrera, F. (2012). SciMAT. Journal of the American Society for Information Science and Technology, 63(8), 1609–1630. https://doi.org/10.1002/asi.22688

  • Donthu, N., & Gustafsson, A. (2020). Effects of COVID-19 on business and research. Journal of Business Research, 117, 284–289. https://doi.org/10.1016/j.jbusres.2020.06.008

  • EPPI-Centre. (2021). Welcome to the EPPI-Centre. EPPI-Centre. https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=63

  • Fisch, C., & Block, J. (2018). Six tips for your (systematic) literature review in business and management research. Management Review Quarterly, 68(2), 103–106. https://doi.org/10.1007/s11301-018-0142-x

  • Gough, D., Oliver, S., & Thomas, J. (2013). Learning from Research. Nesta.

  • Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1(1), 28. https://doi.org/10.1186/2046-4053-1-28

  • Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

  • Harris, J., Quatman, C., Manring, M., Siston, R., & Flanigan, D. (2014). How to write a systematic review. The American Journal of Sports Medicine, 42(11), 2761–2768.

  • Harrison, J. S., Banks, G. C., Pollack, J. M., O’Boyle, E. H., & Short, J. (2017). Publication bias in strategic management research. Journal of Management, 43(2), 400–425.

  • Herrmann, H. (2022). The arcanum of artificial intelligence in enterprise applications: Toward a unified framework. Journal of Engineering and Technology Management, 66(Oct-Dec), 101716. https://doi.org/https://doi.org/10.1016/j.jengtecman.2022.101716

  • Herrmann, H. (2023). What’s next for responsible artificial intelligence: A way forward through responsible innovation. Heliyon, 9(3), e14379. https://doi.org/https://doi.org/10.1016/j.heliyon.2023.e14379

  • Heyvaert, M., Maes, B., & Onghena, P. (2013). Mixed methods research synthesis. Quality & Quantity, 47(2), 659–676. https://doi.org/10.1007/s11135-011-9538-6

  • Hood, W. W., & Wilson, C. S. (2001). The literature of bibliometrics, scientometrics, and informetrics. Scientometrics, 52(2), 291–314.

  • Isoaho, K., Gritsenko, D., & Mäkelä, E. (2021). Topic modeling and text analysis for qualitative policy research. Policy Studies Journal, 49(1), 300–324. https://doi.org/10.1111/psj.12343

  • Jaume-Palasí, L., Matzat, L., Spielkamp, M., & Zweig, K. A. (2021, 21 June 2019). AI Ethics Guidelines Global Inventory. AlgorithmWatch. Retrieved 3 April 2021 from https://algorithmwatch.org/en/ai-ethics-guidelines-global-inventory/

  • Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399.

  • Lee, H., Tamminen, K., Clark, A. M., Slater, L., Spence, J., & Holt, N. (2015). A meta-study of qualitative research examining determinants of children’s independent active free play. International Journal of Behavioral Nutrition and Physical Activity, 12(1), 1–12.

  • Linnenluecke, M. K., Marrone, M., & Singh, A. K. (2020). Conducting systematic literature reviews and bibliometric analyses. Australian Journal of Management, 45(2), 175–194. https://doi.org/10.1177/0312896219877678

  • Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI. Scientometrics, 126(1), 871–906. https://doi.org/10.1007/s11192-020-03690-4

  • Mas-Tur, A., Kraus, S., Brandtner, M., Ewert, R., & Kürsten, W. (2020). Advances in management research. Review of Managerial Science, 14(5), 933–958. https://doi.org/10.1007/s11846-020-00406-z

  • Mas-Tur, A., Modak, N. M., Merigó, J., Roig-Tierno, N., Geraci, M., & Capecchi, V. (2019). Half a century of Quality & Quantity. Quality & Quantity, 53(2), 981–1020.

  • Mingers, J., & Leydesdorff, L. (2015). A review of theory and practice in scientometrics. European Journal of Operational Research, 246(1), 1–19. https://doi.org/10.1016/j.ejor.2015.04.002

  • Müller, V. (2021). Ethics of Artificial Intelligence and Robotics. In E. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Summer 2021 ed.). The Metaphysics Research Lab. https://plato.stanford.edu/entries/ethics-ai/#Oth

  • Ocholla, D. N., & Onyancha, O. B. (2013). The marginalized knowledge: An informetric analysis of indigenous knowledge publications (1990-2004). South African Journal of Libraries and Information Science, 71(3). https://doi.org/10.7553/71-3-593

  • Palmatier, R., Houston, M., & Hulland, J. (2018). Review articles: Purpose, process, and structure. Journal of the Academy of Marketing Science, 46(1), 1–5. https://doi.org/10.1007/s11747-017-0563-4

  • Pfeffer, J., & Sutton, R. (2006). Evidence-based management. Harvard Business Review, 84(1), 62–74. https://www.ncbi.nlm.nih.gov/pubmed/16447370

  • Rousseau, D. (2006). Is there such a thing as “evidence-based management”? Academy of Management Review, 31(2), 256–269.

  • Rynes, S., & Bartunek, J. (2017). Evidence-based management. Annual Review of Organizational Psychology and Organizational Behavior, 4, 235–261.

  • Sandelowski, M., Voils, C., & Barroso, J. (2006). Defining and designing mixed research synthesis studies. Research in the Schools, 13(1), 29.

  • Santana, M., & Cobo, M. J. (2020). What is the future of work? A science mapping analysis. European Management Journal, 38(6), 846–862. https://doi.org/10.1016/j.emj.2020.04.010

  • Sartal, A., González-Loureiro, M., & Vázquez, X. H. (2021). Meta-analyses in management: What can we learn from clinical research? BRQ Business Research Quarterly, 24(1), 91–111. https://doi.org/10.1177/2340944420916310

  • Schiff, D., Rakova, B., Ayesh, A., Fanti, A., & Lennon, M. (2020). Principles to practices for responsible AI. arXiv preprint arXiv:2006.04707.

  • Schlenker, L., & Minhaj, M. (2020). Machine Intelligence and managerial decision-making. In J. Liebowitz (Ed.), Data Analytics & AI. Taylor & Francis.

  • Shneiderman, B. (2021). Responsible AI: Bridging from ethics to practice. Communications of the ACM, 64(8), 32–35.

  • Short, J. (2009). The art of writing a review article. Journal of Management, 35(6), 1312–1317.

  • Siluo, Y., & Qingli, Y. (2017). Are scientometrics, informetrics, and bibliometrics different? Conference: The 16th International Conference on Scientometrics & Informetrics (ISSI2017),

  • Soaita, A., Serin, B., & Preece, J. (2020). A methodological quest for systematic literature mapping. International Journal of Housing Policy, 20(3), 320–343. https://doi.org/10.1080/19491247.2019.1649040

  • Stahl, B., Andreou, A., Brey, P., Hatzakis, T., Kirichenko, A., Macnish, K., Shaelou, S. L., Patel, A., Ryan, M., & Wright, D. (2021). Artificial intelligence for human flourishing. Journal of Business Research, 124, 374–388.

  • Taddeo, M., & Floridi, L. (2018). How AI can be a force for good. Science, 361(6404), 751–752.

  • Tashakkori, A., & Teddlie, C. (2003). The past and future of mixed methods research. In C. Teddlie & A. Tashakkori (Eds), Handbook of Mixed Methods in Social and Behavioral Research (Vol. 1, pp. 671–702). Sage.

  • Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence‐informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222.

  • Wallace, A., Croucher, K., Bevan, M., Jackson, K., O’Malley, L., & Quilgars, D. (2006). Evidence for policy making. Housing Studies, 21(2), 297–314.

  • Wang, Q. (2020). Normalization and Differentiation in Google News. Rutgers.

  • Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards. BMC Medicine, 11(1), 21. https://doi.org/10.1186/1741-7015-11-21

  • Yampolskiy, R. (2013). Artificial intelligence safety engineering. In Philosophy and Theory of Artificial Intelligence (pp. 389–396). Springer.

  • Zhao, Y., & Zhao, R. (2014). Evolution of the development of scientometrics. iConference 2014 Proceedings, Berlin, Germany.

  • Zupic, I., & Čater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429–472. https://doi.org/10.1177/1094428114562629