[Feature] Route sandbox log archive over in-VPC OSS endpoint
- Dominant language
- Python
- Stars
- 485
- Forks
- 81
- Avg merge
- 16h 12m
- Merged PRs (30d)
- 8
Description
## Background
`SandboxLogArchiveTask` runs per-worker as a daily cron job, tars up
each stopped sandbox's log directory and uploads it via `ossutil cp`
to the primary OSS bucket. Both the worker and the OSS bucket live
inside the same VPC, but today the task uses the same `endpoint` the
SDK sees — which has to be the public one because the SDK runs on
developer machines outside the VPC.
Result: every byte of archive traffic (which scales with sandbox
count × log verbosity and is by far the dominant OSS flow) goes over
the public network, paying public-egress bandwidth cost that a VPC
`server_endpoint` would eliminate.
## Proposal
1. Add `server_endpoint: str = ""` to `OssAccountConfig`. Document it
as **server-side-only** — it is deliberately never surfaced in the
STS response returned by `/get_token`, so the SDK cannot depend on
it even accidentally.
2. Have `SandboxLogArchiveTask._archive_one` try `server_endpoint`
first and fall back to `endpoint` on failure. Empty and equal
entries are deduplicated, so a legacy YAML that only sets
`endpoint` still makes exactly one attempt.
Empty `server_endpoint` yields identical behavior to today's config,
so existing YAMLs need no change.
## Scope note
An earlier version of this proposal also introduced a
`POST /stage_sandbox_file` admin API to move the SDK's
`download_via_oss` inline `ossutil cp` (sandbox → OSS leg) to the
server side. That path was **dropped** because:
- The SDK download flow already has a mandatory public-endpoint leg
(OSS → user machine), so optimizing only the sandbox → OSS leg does
not eliminate the public-egress cost — it just cuts it roughly in
half for a low-volume flow.
- The complexity cost was high: a new admin API surface, server-side
ossutil install logic, SDK migration + coordinated release.
- Archive traffic is orders of magnitude larger than user-triggered
`download_file` traffic, so this PR captures ~95% of the benefit at
a fraction of the complexity.
## Non-goals
- SDK upload / download endpoint changes (kept on public endpoint by
design — user machines are outside the VPC).
- Any change to the STS response body.
Contributor guide
No contributing guide indexed for this repository
Research direction
Locate OssAccountConfig, the /get_token response handling, and SandboxLogArchiveTask._archive_one. Check the existing archive upload path and configuration tests first, then verify that server_endpoint remains server-side-only, endpoint attempts are deduplicated with fallback behavior, and legacy configurations still make one upload attempt.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100