rustfs / rustfs/docs.rustfs.com

Docs: add a bilingual On-Demand Migration page under administration/data/bucket

Open
#138 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
MDX
Stars
15
Forks
79
Avg merge
6h 48m
Merged PRs (30d)
6

Description

RustFS has landed On-Demand Migration (ODM): a bucket can name an external S3-compatible source bucket, and a GET for a key that does not exist locally is fetched from that source, streamed to the client, and stored locally in the same pass. It is the RustFS counterpart of Cloudflare R2 Sippy, Tigris shadow buckets, and Alibaba Cloud OSS / Tencent COS mirror-back-to-origin, and it is the answer to the long-standing request for inbound/pull replication from a generic S3 source.

This issue asks for a bilingual (English + Chinese) documentation page for it.

Requested page

  • Path: administration/data/bucket/on-demand-migration (both locales), next to the other bucket-level data-management pages.
  • Source content: docs/operations/on-demand-migration.md in the rustfs/rustfs repository. That guide is the source of truth; the site page should be a user-facing rendering of it, not a fork of its content.

What the page should cover

  1. What ODM is, and how it differs from bucket replication (outbound push), site replication, tiering/ILM, and the replication read-proxy. The positioning table in the guide can be reused directly.
  2. Enabling a source: the RUSTFS_ON_DEMAND_MIGRATION_ENABLED module switch, then the admin API — PUT / PUT ?dry-run=true / GET / DELETE /rustfs/admin/v3/on-demand-migration/{bucket} and GET .../{bucket}/status, with SigV4-signed examples.
  3. The complete configuration reference (source, filter, policy) with defaults and bounds.
  4. Observable semantics: GET/HEAD/LIST/PUT/DELETE, Range GET, versionId, conditional headers, delete markers, SSE-C sources, events, replication, quota and Object Lock interactions, and the x-rustfs-on-demand-migration: source response marker.
  5. Provider presets (s3, aws, minio, rustfs, r2, gcs) and the least-privilege source policy: s3:ListBucket, s3:GetObject, plus s3:GetObjectTagging only when tag copying is enabled.
  6. Integrity and ETag rules, the copied-metadata allowlist, and the provenance metadata written on every pulled object.
  7. Protections: circuit breaker, negative cache, singleflight, concurrency limit, bounded pull queue, bandwidth limit, retries, anti-loop markers.
  8. Error codes (424 SourceUnavailable and its failure classes, the 404 degradation policy, the admin-side codes), observability (the rustfs_on_demand_migration_* metrics, PromQL examples, the status fields), troubleshooting, known limitations, and the security notes — in particular that source credentials live in bucket metadata unencrypted and should be read-only and scoped.

Notes for the writer

  • Never show a real access key or secret in examples; the upstream guide uses AKIA… and REDACTED placeholders.
  • The background backfill job is a separate slice; the guide marks it as such. Please keep the same caveat until it ships, rather than documenting it as available.
  • Two status fields are intentionally null in the current build (breaker.opened_at, served_by_source_ratio); do not describe them as broken.

References

  • Documentation task: rustfs/backlog#2160
  • Tracking plan (semantics table, architecture, merged PR list): rustfs/backlog#2147
  • Original feature request: rustfs/rustfs#6827

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read docs/operations/on-demand-migration.md as the source of truth, then inspect neighboring bucket-level pages under administration/data/bucket and their English and Chinese locale structure. Add the bilingual on-demand-migration pages without forking the guide, preserving its examples, caveats, null status fields, security notes, and referenced behavior. Done means both locales cover the requested sections and render with working navigation and links.

Written by the indexing model from the issue text.

Assessment

Tech stack
markdown
Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.