dense-analysis / dense-analysis/ale

Linting fails on non-UTF-8 files because ale#util#Writefile always writes in Vim internal encoding

Open
#5,139 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Vim Script
Stars
14k
Forks
1.5k
Avg merge
17h 49m
Merged PRs (30d)
1

Description

## Problem

When ALE writes buffer contents to a temporary file (for linting or fixing), `ale#util#Writefile` in `autoload/ale/util.vim` uses Vim's internal encoding (typically UTF-8) without converting to the buffer's `&fileencoding`. This causes encoding mismatch when the file uses a non-UTF-8 encoding such as GB18030, GBK, Shift-JIS, or Latin-1.

## Steps to Reproduce

1. Open a file with a non-UTF-8 encoding, e.g., a Python file with `# -*- coding: GB18030 -*-` header
2. Vim detects and sets `&fileencoding=gb18030`
3. ALE triggers a linter (e.g., flake8, pylint) or fixer that reads from the temp file
4. The temp file is written in UTF-8 (Vim's internal encoding), but the linter expects GB18030 content (matching the file's declared encoding)

## Expected Behavior

The temp file should be written in the buffer's `&fileencoding` so that linters/fixers receive content in the correct encoding.

## Actual Behavior

The temp file is always written in Vim's internal encoding (UTF-8), regardless of the buffer's `&fileencoding`. This causes:
- Linters that respect the file's encoding declaration (e.g., `# -*- coding: GB18030 -*-`) receive UTF-8 bytes but try to decode as GB18030, resulting in `UnicodeDecodeError` or garbled output
- Fixers that process the file may produce corrupted content

## Suggested Fix

In `ale#util#Writefile`, convert buffer lines from `&encoding` to `&fileencoding` using `iconv()` before calling `writefile()`:

```vim
function! ale#util#Writefile(buffer, lines, filename) abort
let l:corrected_lines = getbufvar(a:buffer, '&fileformat') is# 'dos'
\ ? map(copy(a:lines), 'substitute(v:val, ''\r*$'', ''\r'', '''')')
\ : a:lines

" Convert from Vim's internal encoding to the buffer's fileencoding.
let l:fenc = getbufvar(a:buffer, '&fileencoding')

if !empty(l:fenc) && l:fenc !=? &encoding
let l:corrected_lines = map(copy(l:corrected_lines),
\ 'iconv(v:val, &encoding, l:fenc)')
endif

let l:flags = !getbufvar(a:buffer, '&eol') && exists('+fixeol') && !&fixeol ? 'bS' : 'S'

call writefile(l:corrected_lines, a:filename, l:flags) " no-custom-checks
endfunction
```

## Environment

- Vim 9.x / Neovim 0.9+
- OS: Windows / Linux (affects all platforms)
- `&encoding`: utf-8
- `&fileencoding`: gb18030 (or any non-UTF-8 encoding)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.