saltstack / saltstack/salt

[BUG] file.replace open file in binary mode Windows

Open
#65,906 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs-triage
Dominant language
Python
Stars
15.7k
Forks
5.6k
Avg merge
2d 44m
Merged PRs (30d)
80

Description

Description
File replace opens file in binary mode "rb" to see if their is a match first, which results in patterns with start of line and end of line not matching on windows. The code which is not reach, opens the file again without binary on. May also impact utf-8 processing of non ASCII chars.

Thoughts file module.,does it need output encoding option defaulting to the platforms encoding utf-8 or mbcs etc. Does it need a way to enforce a line feed type.

Setup
On Windows Salt 3006.4

On windows create a file with DOS line endings abc.txt but the line "something=" in line in the middle of the file.

replace_dos_file_line:
  file.replace:
    -name: 'abc.txt'
   - pattern: '^(.*)something=\d+'
   - repl: '\1something=123'
   - show_changes: True
   - flags: ['IGNORECASE']

Please be as specific as possible and give set-up details.

  • on-prem machine
  • VM (Virtualbox, KVM, etc. please specify)
  • VM running on a cloud service, please be explicit and add details
  • container (Kubernetes, Docker, containerd, etc. please specify)
  • or a combination, please be explicit
  • jails if it is FreeBSD
  • classic packaging
  • onedir packaging
  • used bootstrap to install

Steps to Reproduce the behavior
See example state above

Expected behavior
That start of line should work in pattern matching

Versions Report
Windows Salt 3006.4
with with salt.utils.files.fopen(path, mode="rb", buffering=bufsize)
Also note that their is a check to see if a file is binary or not, utf-8 file may come across as being binary.

Additional context
Add any other context about the problem here.

Sample file abc.txt

one two
three
four
      something=111111
ten
nine

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing Salt's file.replace implementation and the salt.utils.files.fopen(path, mode="rb", buffering=bufsize) call. Reproduce the Windows DOS-line-ending example with the shown pattern, then inspect the binary-file and encoding checks. Done means start/end-of-line matching works for this case without regressing UTF-8 or binary-file handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.