acl-org / acl-org/acl-anthology

Using Machine Learning to Predict Item Difficulty and Response Time in Medical Tests

Open
#3,614 2 comments 0 reactions 1 assignee Claimed by @anthology-assist View on GitHub
correction metadata waiting
Dominant language
Python
Stars
797
Forks
408
Avg merge
3d 19h
Merged PRs (30d)
36

Description

### Confirm that this is a metadata correction

- [X] I want to file corrections to make the metadata match the PDF file hosted on the ACL Anthology.

### Anthology ID

2024.bea-1.48

### Type of Paper Metadata Correction

- [ ] Paper Title
- [X] Paper Abstract
- [ ] Author Name(s)

### Correction to Paper Title

_No response_

### Correction to Paper Abstract

Prior knowledge of item characteristics, such as difficulty and response time, without pretesting items can substantially save time and cost in high-standard test development. Using a variety of machine learning (ML) algorithms, the present study explored several (non-)linguistic features (such as Coh-Metrix indices) along with MPNet word embeddings to predict the difficulty and response time of a sample of medical test items. In both prediction tasks, the contribution of embeddings to models already containing other features was found to be extremely limited. Moreover, a comparison of feature importance scores across the two prediction tasks revealed that cohesion-based features were the strongest predictors of difficulty, while the prediction of response time was primarily dependent on length-related features.

### Correction to Author Name(s)

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by searching the repository for Anthology ID 2024.bea-1.48, then find the metadata entry containing its abstract. Replace the abstract with the correction text from the issue. Done means the stored metadata for that paper matches the provided abstract, but note the issue is stale and already assigned.

Written by the indexing model from the issue text.

Assessment

Domain
content
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.