UB/crashes with libedit
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 14
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
the code mentions compatibility with libedit, however the following happens in practice (e.g. while using irb from ruby 2.5.0):
- libedit initializes internal copy of rl_outstream
FILE*pointer during rl_initialize()
e = el_init_internal(rl_readline_name, rl_instream, rl_outstream, ...). this saves the current rl_outstream inel->el_outfile.
from that point on, libedit only operates on el_outfile in itsEditLine*context. - in this module, clear_rl_outstream() is called from readline_s_set_output() which fclose()s rl_outstream. then rl_outstream gets a fresh
FILE*assigned - however this does not affect libedit which continues using its el_outfile struct member. - next attempt to write anything to the output file via readline API causes a segfault - at least with musl libc, which is stricter in regards to UB.
probably this was never caught because the developers used glibc where the fclose() FILE structure happened to stay in RAM at the same address. or maybe the new FILE was allocated at the exact same address. it's 100% reproducible using libedit (latest release) and musl libc, and likely when using address sanitizer even on glibc.
it's likely that rl_instream handling has the same bug, but i only investigated the crash i actually got.
arguably it's a bug of libedit to supply rl_outstream but then use it only for initialization - but it appears nobody else was closing the FILE behind libedit's back so far, else it would be fixed in latest libedit version.
(background: i was about to update ruby from 2.5.0 to a newer version in my distro sabotage linux and noticed irb crashing, so i wanted to fix it before switching to a more modern version, so someone who needs to use the old version for whatever reason can go back in commit history and produce a 100% working version).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at readline_s_set_output() and clear_rl_outstream(), then trace how rl_initialize() stores rl_outstream in libedit. Reproduce the crash with libedit and musl libc, and investigate the related rl_instream path mentioned in the report. Done means readline output no longer accesses a closed FILE* and the reported crash is prevented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, ruby
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100