dense-analysis / dense-analysis/ale
Linting fails on non-UTF-8 files because ale#util#Writefile always writes in Vim internal encoding
- Dominant language
- Vim Script
- Stars
- 14k
- Forks
- 1.5k
- Avg merge
- 17h 49m
- Merged PRs (30d)
- 1
Description
## Problem
When ALE writes buffer contents to a temporary file (for linting or fixing), `ale#util#Writefile` in `autoload/ale/util.vim` uses Vim's internal encoding (typically UTF-8) without converting to the buffer's `&fileencoding`. This causes encoding mismatch when the file uses a non-UTF-8 encoding such as GB18030, GBK, Shift-JIS, or Latin-1.
## Steps to Reproduce
1. Open a file with a non-UTF-8 encoding, e.g., a Python file with `# -*- coding: GB18030 -*-` header
2. Vim detects and sets `&fileencoding=gb18030`
3. ALE triggers a linter (e.g., flake8, pylint) or fixer that reads from the temp file
4. The temp file is written in UTF-8 (Vim's internal encoding), but the linter expects GB18030 content (matching the file's declared encoding)
## Expected Behavior
The temp file should be written in the buffer's `&fileencoding` so that linters/fixers receive content in the correct encoding.
## Actual Behavior
The temp file is always written in Vim's internal encoding (UTF-8), regardless of the buffer's `&fileencoding`. This causes:
- Linters that respect the file's encoding declaration (e.g., `# -*- coding: GB18030 -*-`) receive UTF-8 bytes but try to decode as GB18030, resulting in `UnicodeDecodeError` or garbled output
- Fixers that process the file may produce corrupted content
## Suggested Fix
In `ale#util#Writefile`, convert buffer lines from `&encoding` to `&fileencoding` using `iconv()` before calling `writefile()`:
```vim
function! ale#util#Writefile(buffer, lines, filename) abort
let l:corrected_lines = getbufvar(a:buffer, '&fileformat') is# 'dos'
\ ? map(copy(a:lines), 'substitute(v:val, ''\r*$'', ''\r'', '''')')
\ : a:lines
" Convert from Vim's internal encoding to the buffer's fileencoding.
let l:fenc = getbufvar(a:buffer, '&fileencoding')
if !empty(l:fenc) && l:fenc !=? &encoding
let l:corrected_lines = map(copy(l:corrected_lines),
\ 'iconv(v:val, &encoding, l:fenc)')
endif
let l:flags = !getbufvar(a:buffer, '&eol') && exists('+fixeol') && !&fixeol ? 'bS' : 'S'
call writefile(l:corrected_lines, a:filename, l:flags) " no-custom-checks
endfunction
```
## Environment
- Vim 9.x / Neovim 0.9+
- OS: Windows / Linux (affects all platforms)
- `&encoding`: utf-8
- `&fileencoding`: gb18030 (or any non-UTF-8 encoding)
Contributor guide
Assessment
This issue has not been assessed yet.