This title appears in the Scientific Report :
2024
Please use the identifier:
http://dx.doi.org/10.34734/FZJ-2024-01168 in citations.
Pitfalls in using ML to predict executive function performance by linguistic variables
Pitfalls in using ML to predict executive function performance by linguistic variables
Introduction: A connection between executive function (EF) performance and prosody was previously found in numerous mental disorders (Filipe et al., 2018; Le et al., 2011; Nevler et al., 2017). However, it is so far unresolved how different subdomains of EF and prosody are related to each other. Thu...
Saved in:
Personal Name(s): | Kuhles, Gianna (Corresponding author) |
---|---|
Camilleri, Julia / Hamdan, Sami / Heim, Stefan / Eickhoff, Simon / Patil, Kaustubh / Weis, Susanne | |
Contributing Institute: |
Strukturelle und funktionelle Organisation des Gehirns; INM-1 Gehirn & Verhalten; INM-7 |
Imprint: |
2024
|
DOI: |
10.34734/FZJ-2024-01168 |
Conference: | European Workshop on Cognitive Neuropsychology, Bressanone/Brixen (Italy), 2024-01-22 - 2024-01-26 |
Document Type: |
Poster |
Research Program: |
JL SMHB - Joint Lab Supercomputing and Modeling for the Human Brain (JL SMHB-2021-2027) Neuroscientific Data Analytics and AI |
Link: |
OpenAccess |
Publikationsportal JuSER |
Introduction: A connection between executive function (EF) performance and prosody was previously found in numerous mental disorders (Filipe et al., 2018; Le et al., 2011; Nevler et al., 2017). However, it is so far unresolved how different subdomains of EF and prosody are related to each other. Thus, the present study strived to explore the relationships of EF and prosody using a machine learning (ML) regression approach aiming to predict EF performance from various prosodic features.Methods: Healthy participants (n = 231) performed several spontaneous speech tasks, as well as commonly used EF tests, spanning different EF subdomains. Prosodic features were extracted automatically from the speech samples. We then used a standard ML approach to predict EF performance from prosody. As is common, we controlled for confounding effects of age, sex, and education Subsequently, the most predictive features for each of the successfully predicted EF variables were identified.Results: Results indicated that spectral prosodic parameters were particularly important for successful prediction, which is in line with previous literature (Le et al., 2011). However, a thorough assessment of the analysis pipeline revealed a leakage of the effects of sex, age, and education into the prediction, basically indicating the prediction performance – at least for some of the variables – was mainly driven by sex, age, and education confounds, rather than our prosody features. While results of ML analyses might appear to fit with previous results, present findings strongly underline the importance of educated control of any ML pipeline. Thus, we suggest running sanity checks for predicting cognitive performance as well as caution with the interpretation of ML prediction results.Discussion:Taking these methodological considerations into account, the outcome of the present study provides insights into the specific relationships between prosody and executive function performance, concurrently warning about possible pitfalls with the use of ML. While our findings are in line with previous studies (Filipe et al., 2018; Le et al., 2011; Nevler et al., 2017), further research should investigate whether the predictive power of prosody can serve as a biomarker of executive dysfunction in the future. |