Chapter 11: Mixed methods practice in operational research: designs for wicked problems
Open access

In operational research (OR), practitioners model and analyse real-world issues, and support organizational and policy decision-making. Traditionally, they use quantitative methods, such as optimization, simulation, and statistical analysis. Many also use participative, qualitative modelling methodologies known as problem structuring methods (PSMs), which are particularly suitable for wicked problems. There have been recent, renewed calls for OR practice that integrates PSMs with traditional methods, and for guidance for practitioners, students and organizational clients on how to do it and its benefits. Accordingly, in this chapter, I introduce mixed methods practice and theory in OR, review the prevalence of mixing PSMs with traditional OR, and showcase six methodological designs with examples – typical and innovative – from OR interventions across public, private, and third sector organizations. I describe a range of benefits, including successful engagement with dynamic, social, organizational contexts, and discuss current and future research directions.

  • Abuabara, L., & Paucar-Caceres, A. (2021). Surveying applications of strategic options development and analysis (SODA) from 1989 to 2018. European Journal of Operational Research, 292(3), 1051–1065. https://doi.org/10.1016/j.ejor.2020.11.032

  • Ackermann, F., & Eden, C. (2020). Strategic options development and analysis. In M. Reynolds & S. Holwell (Eds), Systems approaches to making change: A practical guide (2nd ed., pp. 139–199). Springer. https://doi.org/10.1007/978-1-4471-7472-1_4

  • Ackermann, F., Franco, L. A., Rouwette, E. A. J. A., & White, L. (2014). Special issue on problem structuring research and practice. EURO Journal on Decision Processes, 2(3–4), 165–172. https://doi.org/10.1007/s40070-014-0037-6

  • Ackermann, F., & Howick, S. (2021). Experiences of mixed method OR practitioners: Moving beyond a technical focus to insights relating to modelling teams. Journal of the Operational Research Society. Advance online publication. https://doi.org/10.1080/01605682.2021.1970486

  • Ackoff, R. L. (1979). The future of operational research is past. Journal of the Operational Research Society, 30(2), 93–104. https://doi.org/10.1057/jors.1979.22

  • Bana e Costa, C. A., Lourenço, J. C., Oliveira, M. D., & Bana e Costa, J. C. (2014). A socio-technical approach for group decision support in public strategic planning: The Pernambuco PPA case. Group Decision and Negotiation, 23, 5–29. https://doi.org/10.1007/s10726-012-9326-2

  • Belton, V., & Stewart, T. (2002). Multiple criteria decision analysis: An integrated approach. Springer. https://doi.org/10.1007/978-1-4615-1495-4

  • Bennett, P. G. (1985). On linking approaches to decision-aiding: Issues and prospects. Journal of the Operational Research Society, 36(8), 659–669. https://doi.org/10.1057/jors.1985.123

  • Brown, J., Cooper, C., & Pidd, M. (2006). A taxing problem: The complementary use of hard and soft OR in the public sector. European Journal of Operational Research, 172(2), 666–679. https://doi.org/10.1016/j.ejor.2004.10.016

  • Checkland, P., & Poulter, J. (2020). Soft systems methodology. In M. Reynolds & S. Holwell (Eds), Systems approaches to making change: A practical guide (2nd ed., pp. 201–253). Springer. https://doi.org/10.1007/978-1-4471-7472-1_5

  • Checkland, P., & Winter, M. (2006). Process and content: Two ways of using SSM. Journal of the Operational Research Society, 57(12), 1435–1441. https://doi.org/10.1057/palgrave.jors.2602118

  • Christie, J. S. (2017). Choosing and combining problem structuring methods in practice. 21st Triennial Conference of the International Federation of Operational Research Societies (IFORS 2017), 17–21 July 2017, Québec City, Canada. https://kar.kent.ac.uk/id/eprint/90277

  • Crowe, S., Brown, K., Tregay, J., Wray, J., Knowles, R., Ridout, D. A., Bull, C., & Utley, M. (2017). Combining qualitative and quantitative operational research methods to inform quality improvement in pathways that span multiple settings. BMJ Quality & Safety, 26(8), 641–652. https://doi.org/10.1136/bmjqs-2016-005636

  • Dyson, R. G., O’Brien, F. A., & Shah, D. B. (2021). Soft OR and practice: The contribution of the founders of OR. Operations Research, 69(3), 727–738. https://doi.org/10.1287/opre.2020.2051

  • Eden, C. (1982). Management science process―Problem construction and the influence of O.R. INFORMS Journal on Applied Analytics, 12(2), 50–60. https://doi.org/10.1287/inte.12.2.50

  • Eden, C. (1989). SODA and cognitive mapping in practice. In J. Rosenhead (Ed.), Rational analysis for a problematic world: Problem structuring methods for complexity, uncertainty and conflict (1st ed., pp. 43–70). Wiley.

  • Eden, C., & Ackermann, F. (2004). Use of “soft OR” models by clients: What do they want from them? In M. Pidd (Ed.), Systems modelling: Theory and practice (pp. 146–163). Wiley.

  • Eden, C., & Radford, J. (Eds). (1990). Tackling strategic problems: The role of group decision support. Sage.

  • Emes, M., Smith, S., Ward, S., & Smith, A. (2019). Improving the patient discharge process: Implementing actions derived from a soft systems methodology study. Health Systems, 8(2), 117–133. https://doi.org/10.1080/20476965.2018.1524405

  • Emes, M., Smith, S., Ward, S., Smith, A., & Ming, T. (2017). Care and flow: Using soft systems methodology to understand tensions in the patient discharge process. Health Systems, 6(3), 260–278. https://doi.org/10.1057/s41306-017-0027-6

  • Friend, J. K., & Hickling, A. (2005). Planning under pressure: The strategic choice approach (3rd ed.). Routledge. https://doi.org/10.4324/9780080480053

  • Gomes Júnior, A. de A., & Schramm, V. B. (2022). Problem structuring methods: A review of advances over the last decade. Systemic Practice and Action Research, 35(1), 55–88. https://doi.org/10.1007/s11213-021-09560-1

  • Hanafizadeh, P., & Mehrabioun, M. (2018). Application of SSM in tackling problematical situations from academicians’ viewpoints. Systemic Practice and Action Research, 31(2), 179–220. https://doi.org/10.1007/s11213-017-9422-y

  • Henao, F., & Franco, L. A. (2016). Unpacking multimethodology: Impacts of a community development intervention. European Journal of Operational Research, 253(3), 681–696. https://doi.org/10.1016/j.ejor.2016.02.044

  • Hillier, F., & Lieberman, G. (2020). Introduction to operations research (11th ed.). McGraw Hill.

  • Hindle, G. A., & Vidgen, R. (2018). Developing a business analytics methodology: A case study in the foodbank sector. European Journal of Operational Research, 268(3), 836–851. https://doi.org/10.1016/j.ejor.2017.06.031

  • Hitchcock, J. H., & Onwuegbuzie, A. J. (2019). Using mathematical formulae as proof for integrating mixed methods research and multiple methods research approaches: A call for multi-mixed methods and meta-methods in a mixed research 2.0 era. International Journal of Multiple Research Approaches, 11(3), 213–234. https://doi.org/10.29034/ijmra.v11n3editorial2

  • Hopes, J. (2021). Reinvigorating soft OR. Inside OR, 601, 8–9. https://www.theorsociety.com/publications/magazines/inside-or/2021-april/

  • Howick, S., & Ackermann, F. (2011). Mixing OR methods in practice: Past, present and future directions. European Journal of Operational Research, 215(3), 503–511. https://doi.org/10.1016/j.ejor.2011.03.013

  • Howick, S., Eden, C., Ackermann, F., & Williams, T. (2008). Building confidence in models for multiple audiences: The modelling cascade. European Journal of Operational Research, 186(3), 1068–1083. https://doi.org/10.1016/j.ejor.2007.02.027

  • International Federation of Operational Research Societies. (n.d.). What is OR? https://www.ifors.org/what-is-or/

  • International Journal of Multiple Research Approaches. (n.d.). Home. Retrieved 20 May 2022, from https://ijmra.org/

  • Jackson, M. C. (2019). Critical systems thinking and the management of complexity. Wiley.

  • Jackson, M. C. (2021). Critical systems practice 2: Produce—Constructing a multimethodological intervention strategy. Systems Research and Behavioral Science, 38(5), 594-609. https://doi.org/10.1002/sres.2809

  • Keeney, R. L. (1992). Value-focused thinking: A path to creative decisionmaking. Harvard.

  • Kotiadis, K., & Mingers, J. (2006). Combining PSMs with hard OR methods: The philosophical and practical challenges. Journal of the Operational Research Society, 57(7), 856–867. https://doi.org/10.1057/palgrave.jors.2602147

  • Liberatore, M. J., & Luo, W. (2013). ASP, the art and science of practice: A comparison of technical and soft skill requirements for analytics and OR professionals. INFORMS Journal on Applied Analytics, 43(2), 194–197. https://doi.org/10.1287/inte.1120.0647

  • Marttunen, M., Lienert, J., & Belton, V. (2017). Structuring problems for multi-criteria decision analysis in practice: A literature review of method combinations. European Journal of Operational Research, 263(1), 1–17. https://doi.org/10.1016/j.ejor.2017.04.041

  • Maxwell, J. A., Chmiel, M., & Rogers, S. E. (2015). Designing integration in multimethod and mixed methods research. In S. N. Hesse-Biber & R. B. Johnson (Eds), The Oxford handbook of multimethods and mixed methods research inquiry (pp. 223–239). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199933624.013.16

  • Maxwell, J. A., & Loomis, D. M. (2003). Mixed methods design: An alternative approach. In A. Tashakkori & C. Teddlie (Eds), Handbook of mixed methods in social and behavioral research (1st ed., pp. 241–272). Sage.

  • Mertens, D. M. (2015). Mixed methods and wicked problems. Journal of Mixed Methods Research, 9(1), 1–4. https://doi.org/10.1177/1558689814562944

  • Midgley, G. (1990). Creative methodology design. Systemist, 12(3), 108–113.

  • Midgley, G. (in press). The systemic intervention approach. In D. Cabrera, L. Cabrera, & G. Midgley (Eds), The Routledge handbook of systems thinking (1st ed.). Routledge. https://www.researchgate.net/publication/349054424_THE_SYSTEMIC_INTERVENTION_APPROACH

  • Midgley, G., Cavana, R. Y., Brocklesby, J., Foote, J. L., Wood, D. R. R., & Ahuriri-Driscoll, A. (2013). Towards a new framework for evaluating systemic problem structuring methods. European Journal of Operational Research, 229(1), 143–154. https://doi.org/10.1016/j.ejor.2013.01.047

  • Mingers, J. (2000). Variety is the spice of life: Combining soft and hard OR/MS methods. International Transactions in Operational Research, 7(6), 673–691. https://doi.org/10.1111/j.1475-3995.2000.tb00224.x

  • Mingers, J. (2001). Combining IS research methods: Towards a pluralist methodology. Information Systems Research, 12(3), 240–259. https://doi.org/10.1287/isre.12.3.240.9709

  • Mingers, J. (2003). A classification of the philosophical assumptions of management science methods. The Journal of the Operational Research Society, 54(6), 559–570. https://doi.org/10.1057/palgrave.jors.2601436

  • Mingers, J., & Brocklesby, J. (1997). Multimethodology: Towards a framework for mixing methodologies. Omega, 25(5), 489–509. https://doi.org/10.1016/S0305-0483(97)00018-2

  • Mingers, J., & Gill, A. (Eds). (1997). Multimethodology: The theory and practice of combining management science methodologies. Wiley.

  • Mingers, J., & Rosenhead, J. (2001). Diverse unity: Looking inward and outward. In J. Rosenhead & J. Mingers (Eds), Rational analysis for a problematic world revisited: Problem structuring methods for complexity, uncertainty and conflict (2nd ed., pp. 337–355). Wiley.

  • Mingers, J., & Rosenhead, J. (2004). Problem structuring methods in action. European Journal of Operational Research, 152(3), 530–554. https://doi.org/10.1016/S0377-2217(03)00056-0

  • Molina-Azorίn, J. F., & Fetters, M. D. (2016). Mixed methods research prevalence studies: Field-specific studies on the state of the art of mixed methods research. Journal of Mixed Methods Research, 10(2), 123–128. https://doi.org/10.1177/1558689816636707

  • Morgan, J. S. (2013). Exploring frameworks for mixing discrete event simulation and system dynamics methods in theory and in practice. (Thesis Identifier: T13553) [Doctoral dissertation, University of Strathclyde]. STAX. https://doi.org/10.48730/h87j-2x52

  • Morgan, J. S., Howick, S., & Belton, V. (2017). A toolkit of designs for mixing discrete event simulation and system dynamics. European Journal of Operational Research, 257(3), 907–918. https://doi.org/10.1016/j.ejor.2016.08.016

  • Munro, I., & Mingers, J. (2002). The use of multimethodology in practice—Results of a survey of practitioners. Journal of the Operational Research Society, 53(4), 369–378. https://doi.org/10.1057/palgrave.jors.2601331

  • Munro, I., & Mingers, J. (2004). Response to Richard Ormerod. Journal of the Operational Research Society, 55(1), 90–93. https://doi.org/10.1057/palgrave.jors.2601670

  • Murray, E. (2021). Modelling in uncertain times. Inside OR, 601, 10. https://www.theorsociety.com/publications/magazines/inside-or/2021-april/

  • Mustafee, N., & Katsaliaki, K. (2020). Classification of the existing knowledge base of OR/MS research and practice (1990–2019) using a proposed classification scheme. Computers and Operations Research, 118, 104920. https://doi.org/10.1016/j.cor.2020.104920

  • Nastasi, B. K., Hitchcock, J. H., & Brown, L. M. (2010). An inclusive framework for conceptualizing mixed methods design typologies: Moving toward fully integrated synergistic research models. In A. Tashakkori & C. Teddlie (Eds), Sage handbook of mixed methods in social & behavioral research (2nd ed., pp. 305–338). Sage. https://doi.org/10.4135/9781506335193.n13

  • Neves, L. P., Dias, L. C., Antunes, C. H., & Martins, A. G. (2009). Structuring an MCDA model using SSM: A case study in energy efficiency. European Journal of Operational Research, 199(3), 834–845. https://doi.org/10.1016/j.ejor.2009.01.053

  • Onwuegbuzie, A. J., & Hitchcock, J. H. (2019). Toward a fully integrated approach to mixed methods research via the 1 + 1 = 1 integration approach: Mixed research 2.0. International Journal of Multiple Research Approaches, 11(1), 7–28. https://doi.org/10.29034/ijmra.v11n1editorial2

  • Ormerod, R. J. (2014). OR competences: The demands of problem structuring methods. EURO Journal on Decision Processes, 2(3–4), 313–340. https://doi.org/10.1007/s40070-013-0021-6

  • Pidd, M. (2004). Complementarity in systems modelling. In M. Pidd (Ed.), Systems modelling: Theory and practice (pp. 1–20). Wiley.

  • Pidd, M. (2009). Tools for thinking: Modelling in management science (3rd ed.). Wiley.

  • Poth, C. N. (2018). The curious case of complexity: Implications for mixed methods research practices. International Journal of Multiple Research Approaches, 10(1), 403–411. https://doi.org/10.29034/ijmra.v10n1a27

  • Ranyard, J. C., Fildes, R., & Hu, T.-I. (2015). Reassessing the scope of OR practice: The influences of problem structuring methods and the analytics movement. European Journal of Operational Research, 245(1), 1–13. https://doi.org/10.1016/j.ejor.2015.01.058

  • Ranyard, J. C., Hopes, J., & Murray, E. (2022). Reinvigorating Soft OR for Practitioners: Report to HORAF. https://www.theorsociety.com/media/6856/reinvigorating-soft-or-for-practitioners-report-to-horaf-v12.pdf

  • Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169. https://doi.org/10.1007/BF01405730

  • Rodríguez-Ulloa, R. A., Montbrun, A., & Martínez-Vicente, S. (2011). Soft system dynamics methodology in action: A study of the problem of citizen insecurity in an Argentinean province. Systemic Practice and Action Research, 24(4), 275–323. https://doi.org/10.1007/s11213-010-9187-z

  • Rosenhead, J. (Ed.). (1989). Rational analysis for a problematic world: Problem structuring methods for complexity, uncertainty and conflict (1st ed.). Wiley.

  • Rosenhead, J. (1992). Into the swamp: The analysis of social issues. Journal of the Operational Research Society, 43(4), 293–305. https://doi.org/10.1057/jors.1992.44

  • Rosenhead, J., & Mingers, J. (2001). A new paradigm of analysis. In J. Rosenhead & J. Mingers (Eds), Rational analysis for a problematic world revisited: Problem structuring methods for complexity, uncertainty and conflict (2nd ed., pp. 1–20). Wiley.

  • Sachdeva, R., Williams, T., & Quigley, J. (2007). Mixing methodologies to enhance the implementation of healthcare operational research. Journal of the Operational Research Society, 58(2), 159–167. https://doi.org/10.1057/palgrave.jors.2602293

  • Sanscartier, M. D. (2020). The craft attitude: Navigating mess in mixed methods research. Journal of Mixed Methods Research, 14(1), 47–62. https://doi.org/10.1177/1558689818816248

  • Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. Jossey-Bass.

  • Schultz, M., & Hatch, M. J. (1996). Living with multiple paradigms: The case of paradigm interplay in organizational culture studies. Academy of Management Review, 21(2), 529–557. https://doi.org/10.5465/AMR.1996.9605060221

  • Smith, C. M., & Shaw, D. (2019). The characteristics of problem structuring methods: A literature review. European Journal of Operational Research, 274(2), 403–416. https://doi.org/10.1016/j.ejor.2018.05.003

  • Tako, A. A., & Kotiadis, K. (2015). PartiSim: A multi-methodology framework to support facilitated simulation modelling in healthcare. European Journal of Operational Research, 244(2), 555–564. https://doi.org/10.1016/j.ejor.2015.01.046

  • Tako, A. A., & Kotiadis, K. (2018). Participative simulation (PartiSim): A facilitated simulation approach for stakeholder engagement. In M. Rabe, A. A. Juan, N. Mustafee, A. Skoogh, S. Jain, & B. Johansson (Eds), Proceedings of the 2018 Winter Simulation Conference (pp. 192–206). IEEE. https://doi.org/10.1109/WSC.2018.8632434

  • Todella, E., & Lami, I. M. (2021). Remote decision-making through a new software combining the strategic choice approach and a multi-criteria decision analysis [Abstract]. 31st European Conference on Operational Research (EURO 2021), Athens, Greece. https://www.euro-online.org/conf/euro31/treat_abstract?paperid=3128

  • Tolk, A., Harper, A., & Mustafee, N. (2021). Hybrid models as transdisciplinary research enablers. European Journal of Operational Research, 291(3), 1075–1090. https://doi.org/10.1016/j.ejor.2020.10.010

  • Tomlinson, R. C. (1984). Rethinking the process of systems analysis and operational research: From practice to precept – and back again. In R. C. Tomlinson & I. Kiss (Eds), Rethinking the process of operational research and systems analysis (pp. 205–223). Pergamon Press.

  • Vennix, J. A. M. (1996). Group model building: Facilitating team learning using system dynamics. Wiley.

  • Willemain, T. R. (1994). Insights on modeling from a dozen experts. Operations Research, 42(2), 213–222. https://doi.org/10.1287/opre.42.2.213

  • Williams, T. (2008). Management science in practice. Wiley.

  • Woolley, R. N., & Pidd, M. (1981). Problem structuring―A literature review. Journal of the Operational Research Society, 32(3), 197–206. https://doi.org/10.1057/jors.1981.42

  • Zolfagharian, M., Romme, A. G. L., & Walrave, B. (2018). Why, when, and how to combine system dynamics with other methods: Towards an evidence-based framework. Journal of Simulation, 12(2), 98–114. https://doi.org/10.1080/17477778.2017.1418639