rustfs / rustfs/docs.rustfs.com
Docs: add a bilingual On-Demand Migration page under administration/data/bucket
Nobody has claimed this yet.
- Dominant language
- MDX
- Stars
- 15
- Forks
- 79
- Avg merge
- 6h 48m
- Merged PRs (30d)
- 6
Description
RustFS has landed On-Demand Migration (ODM): a bucket can name an external S3-compatible source bucket, and a GET for a key that does not exist locally is fetched from that source, streamed to the client, and stored locally in the same pass. It is the RustFS counterpart of Cloudflare R2 Sippy, Tigris shadow buckets, and Alibaba Cloud OSS / Tencent COS mirror-back-to-origin, and it is the answer to the long-standing request for inbound/pull replication from a generic S3 source.
This issue asks for a bilingual (English + Chinese) documentation page for it.
Requested page
- Path:
administration/data/bucket/on-demand-migration(both locales), next to the other bucket-level data-management pages. - Source content:
docs/operations/on-demand-migration.mdin therustfs/rustfsrepository. That guide is the source of truth; the site page should be a user-facing rendering of it, not a fork of its content.
What the page should cover
- What ODM is, and how it differs from bucket replication (outbound push), site replication, tiering/ILM, and the replication read-proxy. The positioning table in the guide can be reused directly.
- Enabling a source: the
RUSTFS_ON_DEMAND_MIGRATION_ENABLEDmodule switch, then the admin API —PUT/PUT ?dry-run=true/GET/DELETE /rustfs/admin/v3/on-demand-migration/{bucket}andGET .../{bucket}/status, with SigV4-signed examples. - The complete configuration reference (
source,filter,policy) with defaults and bounds. - Observable semantics: GET/HEAD/LIST/PUT/DELETE, Range GET,
versionId, conditional headers, delete markers, SSE-C sources, events, replication, quota and Object Lock interactions, and thex-rustfs-on-demand-migration: sourceresponse marker. - Provider presets (
s3,aws,minio,rustfs,r2,gcs) and the least-privilege source policy:s3:ListBucket,s3:GetObject, pluss3:GetObjectTaggingonly when tag copying is enabled. - Integrity and ETag rules, the copied-metadata allowlist, and the provenance metadata written on every pulled object.
- Protections: circuit breaker, negative cache, singleflight, concurrency limit, bounded pull queue, bandwidth limit, retries, anti-loop markers.
- Error codes (424
SourceUnavailableand its failure classes, the 404 degradation policy, the admin-side codes), observability (therustfs_on_demand_migration_*metrics, PromQL examples, the status fields), troubleshooting, known limitations, and the security notes — in particular that source credentials live in bucket metadata unencrypted and should be read-only and scoped.
Notes for the writer
- Never show a real access key or secret in examples; the upstream guide uses
AKIA…andREDACTEDplaceholders. - The background backfill job is a separate slice; the guide marks it as such. Please keep the same caveat until it ships, rather than documenting it as available.
- Two status fields are intentionally
nullin the current build (breaker.opened_at,served_by_source_ratio); do not describe them as broken.
References
- Documentation task: rustfs/backlog#2160
- Tracking plan (semantics table, architecture, merged PR list): rustfs/backlog#2147
- Original feature request: rustfs/rustfs#6827
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read docs/operations/on-demand-migration.md as the source of truth, then inspect neighboring bucket-level pages under administration/data/bucket and their English and Chinese locale structure. Add the bilingual on-demand-migration pages without forking the guide, preserving its examples, caveats, null status fields, security notes, and referenced behavior. Done means both locales cover the requested sections and render with working navigation and links.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- markdown
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100