Hamsa Bastani and Park Sinchaisri, Operations, Information and Decisions, The Wharton School, and Osbert Bastani, Penn Engineering
Abstract: Workers spend a significant amount of time learning how to make good decisions. Evaluating the efficacy of a given decision, however, is quite complicated. For one, decision outcomes are often long-term and relate to the original decision in complex ways. The goal of our paper is to study whether machine learning can be used to infer tips that can help workers learn to make better decisions. Such an algorithm must identify strategies that not only improve worker performance, but that are also interpretable to the human workers so that they can easily understand and follow the tips. We propose a novel machine learning algorithm for inferring interpretable tips that can help users improve their performance in sequential decision-making tasks. We perform a behavioral study to validate our approach. To this end, we designed a virtual kitchen-management game that requires the participant to make a series of decisions to minimize overall service time. Then, we compare the performance of participants shown a tip inferred using our algorithm compared to a control group that is not shown the tip, as well as groups shown either a tip proposed by experienced human workers or a tip inferred by a baseline algorithm. Our experiments show that (i) the tips generated by our algorithm are effective at improving performance, (ii) they significantly outperform the two baseline tips, and (iii) they successfully help participants build on their own experience to discover additional strategies and overcome their resistance to exploring counterintuitive strategies.