micropython / micropython/micropython-lib

tarfile-write issue with file/folder permissions

Open
#797 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
1.1k
Avg merge
7d 6h
Merged PRs (30d)
3

Description

Context

I've been playing with tarfile for a few days and yesterday started trying to create my own archives using tarfile-write installed via mip.

Adding a folder to it works recursively, but transferring the file to the PC via mpremote and expanding it yields all files and folders with --- --- --- permissions (0x000).

Discovery

I investigated and found the source of the issue is related to how the TarInfo object is composed in these lines

in particular on line 102 we find mode to be set to the os.stat(FILE)[0]

tarinfo.mode = stat[0]

stat[0] is either 0x8000 (file) or 0x4000 (folder).
And the subsequently applied masking yields 0x000 as permissions set while adding it to the archive.
I think that mode should be safely set to 0x1a4 > 644 > rw- r-- r--

I have tested by manually composing a TarInfo object to be handed to addfile() rather than patching add()

How to reproduce:
import tarfile
import os

os.mkdir('test_folder')
f = open('test_folder/test_file_01.txt', 'w')
f.write('file 01')
f.close()

os.mkdir('test_folder/sub_folder')

f = open('test_folder/sub_folder/test_file_02.txt', 'w')
f.write('file 02')
f.close()

archive = tarfile.TarFile('test_archive.tar', 'w')
archive.add('test_folder')
archive.close()

Copy the archive from the board to the PC via mpremote cp :test_archive.tar test_archive.tar

Expand the archive and verify the permissions for test_folder.
From Mac os I was not able to chmod 644 test_folder, but using the UI I could unrestrict access.
CleanShot 2024-02-03 at 21 50 23@2x

Looking at the folder content you can verify the same permission issue happens with sub_folder and contained file(s)
CleanShot 2024-02-03 at 21 51 27@2x

Environment:
I run a custom build for ESP32-S3, but this issue is not related to changes I have applied
(name='micropython', version=(1, 23, 0, 'preview'), _machine='LilyGo T-QT Pro with ESP32S3', _mpy=10758)

MicroPython v1.23.0-preview.48.g076516d88.dirty on 2024-01-21; LilyGo T-QT Pro with ESP32S3
Board: ESP32-S3
TarFile version: '0.4.1'

Hope someone can take a look :)
u.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in python-stdlib/tarfile-write/tarfile/write.py at the TarInfo composition around lines 102-106, then run the reproduction using tarfile.TarFile.add() and inspect the extracted archive's permissions. Compare the stat-derived mode with the mode stored in the archive; done means extracted files and folders retain usable permissions rather than 0x000.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.