Experimentally Validated Surveys: Potential for Studying Cognitive and Behavioral Issues in Management

  • Anne-Gaëlle Figureau HYDREOS, Tomblaine, France; and UMR G-EAU, Montpellier, France
  • Anaïs Hamelin LaRGE, EM Strasbourg Business School, Université de Strasbourg, Strasbourg, France
  • Marie Pfiffelmann LaRGE, EM Strasbourg Business School, Université de Strasbourg, Strasbourg, France
Keywords: Methodology, Validity, Experiments, Surveys, Behavior

Abstract

In management, addressing behavioral and cognitive issues empirically is particularly challenging, as it requires combining both internal and external validity with multilevel data collection. The traditional quantitative methodologies used in this area, which are surveys and laboratory experiments, are ill-suited for this purpose. This paper presents a promising, innovative methodology, the experimentally validated survey (EVS), which allows researchers to face this challenge. The EVS relies on the implementation of survey questions previously validated by a controlled experiment. This paper adopts a threefold approach: descriptive, achieved by proposing a structured review of the recent body of literature; practical, achieved by discussing technical issues that one may encounter while running an EVS; and critical, achieved by discussing the advantages and limitations of such methodology for management scholars. Our main contribution is initiating a dynamic movement toward the importation of this innovative methodology into management research.

Downloads

Download data is not yet available.

References


Abell, P., Felin, T. & Foss, N. (2008). Building micro-foundations for the routines, capabilities, and performance links. Managerial and Decision Economics, 29(6), 489–502. doi: 10.1002/mde.1413


Aguinis, H. & Lawal, S. O. (2012). Conducting field experiments using eLancing’s natural environment. Journal of Business Venturing, 27(4), 493–505. doi: 10.1016/j.jbusvent.2012.01.002


Allinson, C. W., Chell, E. & Hayes, J. (2000). Intuition and entrepreneurial behaviour. European Journal of Work and Organizational Psychology, 9(1), 31–43. doi: 10.1080/135943200398049


Barbosa, S., Fayolle, A. & Smith, B. (2019). Biased and overconfident, unbiased but going for it: How framing and anchoring affect the decision to start a new venture. Journal of Business Venturing, 34(3), 528–557. doi: 10.1016/j.jbusvent.2018.12.006


Barsade, S. G. (2002). The ripple effect: Emotional contagion and its influence on group behavior. Administrative Science Quarterly, 47(4), 644–675. doi: 10.2307/3094912


Bekir, I., El Harbi, S., Grolleau, G. & Sutan, A. (2015). Efficiency, equality, positionality: What do people maximize? Experimental vs. hypothetical evidence from Tunisia. Journal of Economic Psychology, 47(C), 77–84. doi: 10.1016/j.joep.2015.01.007


Bird, B. & Schjoedt, L. (2009) Entrepreneurial behavior: Its nature, scope, recent research, and agenda for future research. In A. L. Carsrud & M. Brännback (Eds.), Understanding the entrepreneurial mind (pp. 327–358). Springer. doi: 10.1007/978-1-4419-0443-0_15


Bird, B., Schjoedt, L. & Baum, J. R. (2012). Editor’s introduction. Entrepreneurs’ behavior: Zlucidation and measurement. Entrepreneurship, Theory and Practice, 36(5), 889–913. doi: 10.1111/j.1540-6520.2012.00535.x


Bohm, P. (1972). Estimating demand for public goods: An experiment. European Economic Review, 3(2), 111–130. doi: 10.1016/0014-2921(72)90001-3


Busenitz, L. W. (1999). Entrepreneurial risk and strategic decision making: It’s a matter of perspective. Journal of Applied Behavioral Science, 35(3), 325–340. doi: 10.1177/0021886399353005


Busenitz, L. W. & Barney, J. B. (1997). Differences between entrepreneurs and managers in large organizations: Biases and heuristics in strategic decision-making. Journal of Business Venturing, 12(1), 9–30. doi: 10.1016/s0883-9026(96)00003-1


Caliendo, M., Fossen, F. M. & Kritikos, A. S. (2009). Risk attitudes of nascent entrepreneurs–new evidence from an experimentally validated survey. Small Business Economics, 32, 153–167. doi: 10.1007/s11187-007-9078-6


Campbell, D. T. & Stanley, J. C. (1966) Experimental and quasi-experimental designs for research. Houghton Mifflin Company. doi: 10.4135/9781483346229.n137


Chandler, G. N. & Lyon, D. W. (2001). Issues of research design and construct measurement in entrepreneurship research: The past decade. Entrepreneurship, Theory and Practice, 25(4), 101–114. doi: 10.1177/104225870102500407


Chang, J. B., Lusk, J. L. & Norwood, F. B. (2009). How closely do hypothetical surveys and laboratory experiments predict field behavior? American Journal of Agricultural Economics, 91(2), 518–534. doi: 10.1111/j.1467-8276.2008.01242.x


Charness, G. (2010). Laboratory experiments: Challenges and promise: A review of “Theory and Experiment: What are the Questions?” by Vernon Smith. Journal of Economic Behavior & Organization, 73(1), 21–23. doi: 10.1016/j.jebo.2008.11.005


Charness, G., Gneezy, U. & Kuhn, M. A. (2013). Experimental methods: Extra-laboratory experiments-extending the reach of experimental economics. Journal of Economic Behavior & Organization, 91, 93–100. doi: 10.1016/j.jebo.2013.04.002


Chen, S. S., Ho, K. Y. & Ho, P. H. (2014). CEO overconfidence and long-term performance following R&D increases. Financial Management, 43(2), 245–269. doi: 10.1111/fima.12035


Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64–73. doi: 10.2307/3150876


Coppola, M. (2014). Eliciting risk-preferences in socio-economic surveys: How do different measures perform? The Journal of Socio-Economics, 48, 1–10. doi: 10.1016/j.socec.2013.08.010


Croson, R., Anand, J. & Agarwal, R. (2007). Using experiments in corporate strategy research. European Management Review, 4(3), 173–181. doi: 10.1057/palgrave.emr.1500082


Crowne, D. P. & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24(4), 349–354. doi: 10.1037/h0047358


Ding X., Hartog, J. & Sun, Y. (2010). Can we measure individual risk attitudes in a survey? IZA working paper, n°4807.


Dohmen T., Falk, A., Huffman, D. & Sunde, U. (2005). Explaining trust: Disentangling the roles of beliefs and risk preference. IZA Working Paper.


Dohmen, T., Falk, A., Huffman, D., Sunde, U. et al. (2011). Individual risk attitudes: Measurement, determinants, and behavioral consequences. Journal of the European Economic Association, 9(3), 522–550. doi: 10.1111/j.1542-4774.2011.01015.x


Douglas, E. J. & Shepherd, D. A. (2000). Entrepreneurship as a utility maximizing response. Journal of Business Venturing, 15(3), 231–251. doi: 10.1016/s0883-9026(98)00008-1


Falk, A., Becker, A., Dohmen, T. J., Huffman, D. & Sunde, U. (2016). The preference survey module: A validated instrument for measuring risk, time, and social preferences. SSRN Electronic Journal, 1–66. doi: 10.2139/ssrn.2725874


Falk, A. & Fehr, E. (2003). Why labour market experiments? Labour Economics, 10(4), 399–406. doi: 10.1016/s0927-5371(03)00050-2


Fehr, E., Fischbacher, U., Von Rosenbladt, B., Schupp, J. et al. (2002). A nation-wide laboratory: Examining trust and trustworthiness by integrating behavioral experiments into representative surveys. SSRN Electronic Journal, 1–23. doi: 10.2139/ssrn.385120


Felin, T., Foss, N. J. & Ployhart, R. E (2015). The microfoundations movement in strategy and organization theory. The Academy of Management Annals, 9(1), 575–632. doi: 10.1080/19416520.2015.1007651


Galasso, A. & Simcoe, T. S. (2011). CEO overconfidence and innovation. Management Science, 5(8), 1469–1484. doi: 10.1287/mnsc.1110.1374


Gillespie, R. (1991) Manufacturing knowledge: A history of the Hawthorne experiments. Cambridge University Press.


Gneezy, U. & Imas, A. (2017) Lab in the field: Measuring preferences in the wild. In A. Banerjee & E. Duflo (Eds.), Handbook of field experiments (vol. 1, pp. 439–464). Elsevier. doi: 10.1016/bs.hefe.2016.08.003


Goldstein, W. M. & Hogarth, R. M. (1997). Judgment and decision research: Some historical context. In W. M. Goldstein & R. M. Hogarth (Eds.), Cambridge series on judgment and decision making. Research on judgment and decision making: Currents, connections, and controversies (pp. 3–65). Cambridge University Press. doi: 10.1002/9780470752937


Graf, L., König, A., Enders, A. & Hungenberg, H. (2012). Debiasing competitive irrationality: How managers can be prevented from trading off absolute for relative profit. European Management Journal, 30(4), 386–403. doi: 10.1016/j.emj.2011.12.001


Grazzini, F. & Boissin, J.-P. (2013). Managers’ mental models of small business acquisition; the case of the SME French Transfert market. M@n@gement, 16(1), 49–87. doi: 10.3917/mana.161.0049


Grégoire, D., Binder, J. & Rauch, A. (2019). Navigating the validity tradeoffs of entrepreneurship research experiments: A systematic review and best-practice suggestions. Journal of Business Venturing, 34(2), 284–310. doi: 10.1016/j.jbusvent.2018.10.002


Hainmueller, J., Hangartner, D. & Yamamoto, T. (2015). Validating vignette and conjoint survey experiments against real-world behavior. Proceedings of the National Academy of Sciences, 112, 2395–2400. doi: 10.1073/pnas.1416587112


Hardeweg, B., Menkhoff, L. & Waibel, H. (2013). Experimentally validated survey evidence on individual risk attitudes in rural Thailand. Economic Development and Cultural Change, 61(4), 859–888. doi: 10.1086/670378


Harrison, G. W. & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055. doi: 10.1257/0022051043004577


Hisrich, R., Langan-Fox, J. & Grant, S. (2007). Entrepreneurship research and practice: A call to action for psychology. American Psychologist, 62(6), 575–589. doi: 10.1037/0003-066x.62.6.575


Hogarth, R. M. (1987). Judgement and choice: The psychology of decision (2nd ed.). John Wiley.


Huber, L. R., Sloof, R. & Van Praag, M. (2014). The effect of early entrepreneurship education: Evidence from a field experiment. European Economic Review, 72(C), 76–97. doi: 10.1016/j.euroecorev.2014.09.002


James, H. (2006). Family capitalism: Wendels, Haniels, Falcks, and the continental European model. Harvard University Press.


Johnson, N. D. & Mislin, A. (2012). How much should we trust the world values survey trust question? Economics Letters, 116(2), 210–212. doi: 10.1016/j.econlet.2012.02.010


Kahneman, D. (2003). Maps of bounded rationality: Psychology for behavioral economics. American Economic Review, 93(5), 1449–1475. doi: 10.1257/000282803322655392


Kahneman, D., Slovic, P. & Tversky, A. (1982), Judgement under uncertainty: Heuristics and biases. Cambridge University Press. doi: 10.1017/cbo9780511809477


Kirk, R. (1995). Experimental design, procedures for the behavioural sciences. Brooks/Cole Publishing Company.


Laroche, H. & Nioche, J. P. (2015). L’approche cognitive de la stratégie d’entreprise. Revue Française de Gestion, 8(253), 97–120. doi: 10.3166/rfg.160.81-108


Lavrakas P. J. (2008). Encyclopedia of survey research methods. Sage. doi: 10.4135/9781412963947


List, J., Sadoff, A. & Wagner, M. (2011). So you want to run and experiment, now what? Some simple rules of thumb for optimal experimental design. Experimental Economics, 14(4), 439–457. doi: 10.1007/s10683-011-9275-7


McNeil, B., Pauker, S., Sox, H. & Tversky, A. (1982). On the elicitation of preferences for alternative therapies. The New England Journal of Medicine, 306(21), 1259–1262. doi: 10.1056/nejm198205273062103


Mintzberg, H. (1973). The nature of managerial work. Harper & Row.


Mitchell, R., Busenitz, L., Bird, B., Caglio, C. et al. (2007). The central question in entrepreneurial cognition research 2007. Entrepreneurship Theory and Practice, 31(1), 1–27. doi: 10.1111/j.1540-6520.2007.00161.x


Mitchell, R., Busenitz, L., Lant, T., McDougall, P. et al. (2002). Toward a theory of entrepreneurial cognition: Rethinking the people side of entrepreneurship research. Entrepreneurship, Theory and Practice, 27(2), 93–104. doi: 10.1111/1540-8520.00001


Mizik, N. (2010). The theory and practice of myopic management. Journal of Marketing Research, 47(4), 594–611. doi: 10.1509/jmkr.47.4.594


Molina-Azorin, J. F., Lopez-Gamero, M. D., Pereira-Moliner, J. & Pertusa-Ortega, E. M. (2012). Mixed methods studies in entrepreneurship research: Applications and contributions. Entrepreneurship and Regional Development, 24(5–6), 425–456. doi: 10.1080/08985626.2011.603363


Moore, D. & Flynn, F. J. (2008). The case for behavioral decision research in organizational behavior. The Academy of Management Annals, 2(1), 399–431. doi: 10.1080/19416520802211636


Moore, D. A. & Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502–517. doi: 10.1037/0033-295x.115.2.502


Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver & L. S. Wrightsman (Eds.), Measures of social psychological attitudes (vol. 1, pp. 17–59). Academic Press. doi: 10.1016/b978-0-12-590241-0.50006-x


Phan, P. & Wright, M. (2018). Advancing the science of human cognition and behavior. Academy of Management Perspectives, 32(2), 179–181. doi: 10.5465/amp.2018.0058


Powell, T., Lovallo, D. & Fox, C. (2011). Behavioral strategy. Strategic Management Journal, 32(13), 1369–1386. doi: 10.1002/smj.968


Sadoff, S. & Samek, A. (2019). Can interventions affect commitment demand? A field experiment on food choice. Journal of Economic Behavior & Organization, 158(C), 90–109. doi: 10.1016/j.jebo.2018.11.016


Shepherd, D.A., Williams, T. A. & Patzelt, H. (2015). Thinking about entrepreneurial decision making: Review and research agenda. Journal of Management, 41(1), 11–46. doi: 10.1177/0149206314541153


Simon, H. A. (1947). Administrative behavior. Macmillan.


Simon, H. A (1955). A behavioral model of rational choice. Quarterly Journal of Economics, 69(1), 99–118. doi: 10.2307/1884852


Simon, H. A. (1979). Rational decision making in business organizations. American Economic Review, 69(4), 493–513.


Song, Y., Liu Y., Lanaj, M. W. K., Johnson, R. E. et al. (2018). A social mindfulness approach to understanding experiences customer mistreatment: A within person field experiment. Academy of Management Journal, 61(3), 994–1020. doi: 10.5465/amj.2016.0448


Stein, J. C. (1988). Takeover threats and managerial myopia. Journal of Political Economy, 96(1), 61–80. doi: 10.1086/261524


Stein, J. C. (1989). Efficient capital markets, inefficient firms: A model of myopic corporate behavior. Quarterly Journal of Economics, 104(4), 655–669. doi: 10.2307/2937861


Thaler, R. & Sunstein C. R. (2008). Nudge: Improving decisions about health, wealth, and happiness. Yale University Press.


Tashakkori, A. & Teddlie, C. (2003). Handbook of mixed methods in social & behavioral research. Sage.


Vieider, F., Lefebvre, M., Bouchouicha, R., Chmura, T. et al. (2015). Common components of risk and uncertainty attitudes across contexts and domains: Evidence from 30 countries. Journal of the European Economic Association, 13(3), 421–452. doi: 10.1111/jeea.12102


Vischer, T., Dohmen, T., Falk, A., Huffman, D. et al. (2013). Validating an ultra-short survey measure of patience. Economics Letters, 120(2), 142–145. doi: 10.1016/j.econlet.2013.04.007


Vo, L.-C., Culié, J.-D. & Mounoud, E. (2016). Microfoundations of decoupling: From a coping theory perspective. M@n@gement, 19(4), 248–276. doi: 10.3917/mana.194.0248


Williams, D., Wood, M., Mitchell, R. & Urbig, D. (2019). Applying experimental methods to advance entrepreneurship research: On the need for and publication of experiments. Journal of Business Venturing, 34(2), 215–223. doi: 10.1016/j.jbusvent.2018.12.003


Zellmer-Bruhn, M., Caligiuri, P. & Thomas, D. C. (2016). From the editors: Experimental designs in international business research. Journal of International Business Studies, 47(4), 399–407. doi: 10.1057/jibs.2016.12


Zikmund, G., Babin, B. J., Carr, J. C. & Griffin, M. (2013). Business research methods. Cengage Learning Custom.

Published
2020-12-16
How to Cite
Figureau, A.-G., Hamelin, A., & Pfiffelmann, M. (2020). Experimentally Validated Surveys: Potential for Studying Cognitive and Behavioral Issues in Management. M@n@gement, 23(4), 1–12. https://doi.org/10.37725/mgmt.v23i4.5613
Section
Original Research Articles