Running Behavioral Operations Experiments Using Amazon's Mechanical Turk
- Authors
- Lee, Yun Shin; Seo, Yong Won; Siemsen, Enno
- Issue Date
- May-2018
- Publisher
- WILEY
- Keywords
- behavioral operations; newsvendor decisions; procurement auction; supply chain contracting; MTurk
- Citation
- PRODUCTION AND OPERATIONS MANAGEMENT, v.27, no.5, pp 973 - 989
- Pages
- 17
- Journal Title
- PRODUCTION AND OPERATIONS MANAGEMENT
- Volume
- 27
- Number
- 5
- Start Page
- 973
- End Page
- 989
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/2215
- DOI
- 10.1111/poms.12841
- ISSN
- 1059-1478
1937-5956
- Abstract
- Mechanical Turk (MTurk), an online labor market run by Amazon, provides a web platform for conducting behavioral experiments; the site offers immediate and inexpensive access to a large subject pool. In this study, we review recent research about using MTurk for behavioral experiments and test the validity of using MTurk for experiments in behavioral operations management. We recruited subjects from MTurk to replicate the inventory management experiment from Bolton and Katok (), as well as the procurement auction experiment from Engelbrecht-Wiggans and Katok (), and the supply chain contracting experiment from Loch and Wu (). We successfully replicate individual biases in the inventory management and procurement auction experiments, but learning in the individual tasks occurs more slowly on MTurk compared to the original studies. Further, we find that social preference manipulations in the supply chain experiment are ineffective in changing the behavior of MTurk subjects, in contrast to the original study. We conducted an additional replication study of the supply chain contracting experiment using student subjects in a standard laboratory. Results from this laboratory replication also fail to replicate the original laboratory study, indicating that the effect of social preferences on supply chain contracting may not be robust to alternative subject pools. We conclude that factors potentially influencing the differences observed on MTurk are less related to the online environment, but more related to the diversity and characteristics of subject pool on MTurk. Overall, MTurk appears to be an important and relevant tool for researchers in behavioral operations, but we caution researchers about slower learning of the MTurk subjects and the use of social preference manipulations on MTurk.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Business & Economics > School of Business Administration > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/2215)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.