inveniosoftware / inveniosoftware/rfcs

[Proposal] A cleaner and clearer files model and interface

Open
#37 0 comments 1 reaction 1 assignee Assigned to @lnielsen View on GitHub
Proposal: Accepted
Dominant language
No language data
Stars
4
Forks
22
PR merge metrics
No merged PRs in 30d

Description

### Motivation

The current implementation of invenio-files-rest is in places confusing to follow and doesn't provide an easy-to-use Python API or allow for easy extensibility to support multiple storage backends in one system.

See also related discussion on https://github.com/inveniosoftware/invenio-files-rest/issues/255.

### Summary

Proposed changes

* Storage backends are registered using entry points, and a(n overridable) name→class mapping added to config
* `FileInstance.storage_class` is migrated from `String(1)` to a larger string, and used to point to the storage backend entry point name used for the file. It is currently unused.
* A new storage factory is created that uses `FileInstance.storage_class` to mediate access to stored files based on `.storage_class`
* The new storage factory is the default for new cookiecutters, but the old pyfs storage factory remains with the old behaviour and a DeprecationWarning, to provide a migration path
* The new storage factory can be overridden to use another way to determine storage class for both existing files and new files. I expect to make this a class-based factory, with obviously overrideable methods.
* An alembic migration that ALTERs `FileInstance.storage_class` to extend its length and populates it with according to some (yet to be determined) heuristic
* `FileInstance.storage(**kwargs)` has its `**kwargs` deprecated. Possibly deprecate in favour of a (cached) `file` property, for a cleaner interface. New interface is e.g. `with file_instance.file.open() as f:`
* Update documentation to reflect changes
* Updates to invenio-s3 to use the new storage framework implementation

### Resources

Cottage Labs is offering 2 days a week of developer effort that could be spent on this. I haven't yet estimated the amount of effort required.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.