Support for Cloudflare's D1 and/or R2
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 931
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 4
Description
## Feature Description
Cloudflare has been developing an alternative to traditional storage providers, with the benefit that it has no egress fees for the data.
- [**D1**](https://developers.cloudflare.com/d1) is a relational database build on SQLite, which currently is on Alpha. It can be access and manipulated through the online GUI on [Cloudflare's Dashboard](https://dash.cloudflare.com/) or through the CLI using a npm package called [wrangler](https://github.com/cloudflare/workers-sdk/tree/main/packages/wrangler).
- [**R2**](https://developers.cloudflare.com/r2) is the Cloudflare's implementation of an object storage service. It can be used through the GUI on [Cloudflare's Dashboard](https://dash.cloudflare.com/), through the CLI with [wrangler](https://github.com/cloudflare/workers-sdk/tree/main/packages/wrangler), or using the [Amazon's S3 API](https://aws.amazon.com/s3/).
## Proposal
1. First the easiest, R2. To begin with, test if Apache Libcloud [`libcloud.storage.drivers.s3`](https://libcloud.readthedocs.io/en/stable/apidocs/libcloud.storage.drivers.html#module-libcloud.storage.drivers.s3) works correctly with an R2 bucket, and if so, update the documentation to reflect that said module can be used with R2. It would be a good idea to document what functions are NOT available, according to Cloudflare's R2 [documentation](https://developers.cloudflare.com/r2/data-access/s3-api/api/).
2. If `libcloud.storage.drivers.s3` doesn't work with R2, another module can be created (`libcloud.storage.drivers.r2`, for example), which is based on the S3's module but with a few adjustments, and removing functions that are unavailable on R2. I do think this is a cleaner solution, to keep S3 separated from R2.
3. Finally, as Apache Libcloud does not (yet) manage Cloud Databases, a fine addition to the library would be to start a new module (maybe `libcloud.sql.*`?) and implement certain popular SQL Databases, such as [Amazon's RDS](https://aws.amazon.com/rds/), [Azure's SQL Database](https://azure.microsoft.com/en-us/products/azure-sql/database/), and of course Cloudflare's D1.
Contributor guide
Research direction
Start by reviewing libcloud.storage.drivers.s3 against Cloudflare R2's S3 API documentation and determine whether existing compatibility can be documented or a separate libcloud.storage.drivers.r2 module is needed. The broader proposal also suggests designing libcloud.sql.* for D1 and other managed SQL databases; done requires an agreed scope, implementation, tests, and documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, sql, sqlite
- Domain
- cloud, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100