Using Deep Learning to Detect Facial Markers of Complex Decision Making
In: Guglielmo, 2022
academicJournal
Zugriff:
In this paper, we report on an experiment with The Walking Dead (TWD), which is a narrative-driven adventure game where players have to survive in a post-apocalyptic world filled with zombies. We used OpenFace software to extract action unit (AU) intensities of facial expressions characteristic of decision-making processes and then we implemented a simple convolution neural network (CNN) to see which AUs are predictive of decision-making. More specifically, this study aims to identify the facial regions that are predictive of decision-making. Our results provide evidence that the pre-decision variations in action units 17 (chin raiser), 23 (lip tightener), and 25 (parting of lips) are predictive of decision-making processes. Furthermore, when combined, their predictive power increased up to .81 accuracy on the test set; we offer speculations about why it is that these particular three AUs were found to be connected to decision-making. Our results also suggest that machine learning methods in combination with video games may be used to accurately and automatically identify complex decision-making processes using AU intensity alone. Finally, our study offers a new method to test specific hypotheses about the relationships between higher-order cognitive processes and behavior, which relies on both narrative video games and easily accessible software, like OpenFace.
Titel: |
Using Deep Learning to Detect Facial Markers of Complex Decision Making
|
---|---|
Autor/in / Beteiligte Person: | Guglielmo, Gianluca ; Font Peradejordi, Irene ; Klincewicz, Michał ; Browne, C. ; Kishimoto, A. ; Schaeffer, J. |
Link: | |
Zeitschrift: | Guglielmo, 2022 |
Veröffentlichung: | Springer, 2022 |
Medientyp: | academicJournal |
ISBN: | 978-3-031-11487-8 (print) ; 3-031-11487-6 (print) |
Schlagwort: |
|
Sonstiges: |
|