acl-org / acl-org/acl-anthology
Using Machine Learning to Predict Item Difficulty and Response Time in Medical Tests
- Dominant language
- Python
- Stars
- 797
- Forks
- 408
- Avg merge
- 3d 19h
- Merged PRs (30d)
- 36
Description
### Confirm that this is a metadata correction
- [X] I want to file corrections to make the metadata match the PDF file hosted on the ACL Anthology.
### Anthology ID
2024.bea-1.48
### Type of Paper Metadata Correction
- [ ] Paper Title
- [X] Paper Abstract
- [ ] Author Name(s)
### Correction to Paper Title
_No response_
### Correction to Paper Abstract
Prior knowledge of item characteristics, such as difficulty and response time, without pretesting items can substantially save time and cost in high-standard test development. Using a variety of machine learning (ML) algorithms, the present study explored several (non-)linguistic features (such as Coh-Metrix indices) along with MPNet word embeddings to predict the difficulty and response time of a sample of medical test items. In both prediction tasks, the contribution of embeddings to models already containing other features was found to be extremely limited. Moreover, a comparison of feature importance scores across the two prediction tasks revealed that cohesion-based features were the strongest predictors of difficulty, while the prediction of response time was primarily dependent on length-related features.
### Correction to Author Name(s)
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by searching the repository for Anthology ID 2024.bea-1.48, then find the metadata entry containing its abstract. Replace the abstract with the correction text from the issue. Done means the stored metadata for that paper matches the provided abstract, but note the issue is stale and already assigned.
Written by the indexing model from the issue text.
Assessment
- Domain
- content
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100