The source_hash parameter for file.managed doesn't support files of arbitrary (utf16) encoding
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 5.6k
- Avg merge
- 2d 44m
- Merged PRs (30d)
- 80
Description
So, Lol. I wouldn't have thought this as being too common for it to be too big of a deal, but it turns out that the hash file for y'alls windows installer (https://repo.saltstack.com/windows/Salt-Minion-2018.3.3-Py2-x86-Setup.exe.md5) is a utf16 encoded md5. Due to it being utf16, the file.managed fails.
Here's a statefile:
Download Salt-Minion installer for windows:
file.managed:
- source: https://repo.saltstack.com/windows/Salt-Minion-2018.3.3-Py2-x86-Setup.exe
- name: /path/to/wherever
- source_hash: https://repo.saltstack.com/windows/Salt-Minion-2018.3.3-Py2-x86-Setup.exe.md5
- mode: 0664
This results in:
vmware-iso: ----------
vmware-iso: ID: Download Salt-Minion installer for windows
vmware-iso: Function: file.managed
vmware-iso: Name: /path/to/wherever
vmware-iso: Result: False
vmware-iso: Comment: Unable to manage file: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
vmware-iso: Started: 10:52:51.086256
vmware-iso: Duration: 326.789 ms
vmware-iso: Changes:
vmware-iso:
This is in the 2019.2 branch. If you look in salt.modules.file, you can see that get_managed() will call salt.utils.hashutils.get_hash() in order to process the source_hash file. The salt.utils.hashutils.get_hash() function, which doesn't take any wildarg or kwarg parameters, then calls salt.utils.files.fopen() to open the file. Although salt.utils.files.fopen() function can take an encoding= parameter, the get_hash() function doesn't even bother since it only has three positional parameters and none of them are an encoding.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in salt.modules.file.get_managed() and follow its call to salt.utils.hashutils.get_hash(); then inspect how salt.utils.files.fopen() handles encoding. Done means file.managed can process a UTF-16 source_hash file without the reported decode error, with the relevant behavior covered by tests if the repository has them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100