fsspec / fsspec/filesystem_spec

Standardize content_type handling when writing files

Open
#916 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.4k
Forks
490
Avg merge
2d 3h
Merged PRs (30d)
38

Description

Would it be helpful to standardize the ability to set the content type of files written by fsspec? Several backends have / are requesting the ability to set this (https://github.com/fsspec/s3fs/blob/4f289eaa34dfe8337a72f5a0148c41a44793fde0/s3fs/core.py#L971-L974, https://github.com/fsspec/adlfs/issues/294). Each backend will typically have a different "native" was for setting things. With S3 it's a ContentType keyword. With azure, it's content_settings=azure.storage.blob.ContentSettings(content_type=...).

This proposal would make content_type a proper keyword of pipe, put, (and any others writing data). The backend would be responsible for setting it appropriately.

We might also need to standardize a default behavior. s3fs uses a library to guess the content type. I'm not sure if that's appropriate (if it were, I'd think that boto / azure-storage-blob would do it?).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the proposed pipe and put keyword handling in fsspec, then compare the linked s3fs/core.py implementation with the adlfs issue. Determine which write methods need the standardized content_type keyword and whether a default behavior is required; done means the API decision and backend responsibilities are documented and consistently specified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.