Using Machine Learning to Predict Item Difficulty and Response Time in Medical Tests

### Confirm that this is a metadata correction

- [X] I want to file corrections to make the metadata match the PDF file hosted on the ACL Anthology.

### Anthology ID

2024.bea-1.48

### Type of Paper Metadata Correction

- [ ] Paper Title
- [X] Paper Abstract
- [ ] Author Name(s)

### Correction to Paper Title

_No response_

### Correction to Paper Abstract

Prior knowledge of item characteristics, such as difficulty and response time, without pretesting items can substantially save time and cost in high-standard test development. Using a variety of machine learning (ML) algorithms, the present study explored several (non-)linguistic features (such as Coh-Metrix indices) along with MPNet word embeddings to predict the difficulty and response time of a sample of medical test items. In both prediction tasks, the contribution of embeddings to models already containing other features was found to be extremely limited. Moreover, a comparison of feature importance scores across the two prediction tasks revealed that cohesion-based features were the strongest predictors of difficulty, while the prediction of response time was primarily dependent on length-related features.

### Correction to Author Name(s)

_No response_