GoogleCloudPlatform / GoogleCloudPlatform/gsutil

Support for extended attributes

Open
#965 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

Most filesystems and operating systems these days support extended attributes (Linux, FreeBSD, MacOS, & Windows all do) . It would be nice if `gsutil cp` and `gsutil rsync` were able to preserve these attributes when uploading & downloading.

And to make sure there's no confusion, I'm not referring to POSIX attributes, which are already supported, but extended attributes, which are arbitrary key/value pairs.

For example on linux I can do `setfattr -n user.foo -v bar myfile.txt` to set `user.foo=bar` on `myfile.txt`. The idea is that when this file is copied to google storage, these attributes would be copied as custom object metadata.

Note though, that on some implementations of file extended attributes, the attributes have to be namespaced. For example on linux you have 4 namespaces, `security`, `system`, `trusted`, and `user`, hence why in the above example, it was `user.foo`. Thus to support this, `gsutil` will need to pick a specific namespace to map metadata into. Meaning if an object in google storage has the key `foo`, you can't just set the extended attribute `foo`, it has to be `user.foo`. And since this mapping has to apply when downloading objects, I would recommend that the `user.` prefix be stripped off when uploading.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the gsutil cp and gsutil rsync entry points and how they currently handle POSIX attributes and custom object metadata. Define the namespace mapping for extended attributes, including the proposed user. prefix behavior, then verify that uploads and downloads preserve the intended key/value pairs across the supported operating systems.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.