SignatureDoesNotMatch issue with s3-presign
まだ誰も着手していません。
- 主要言語
- Rust
- スター
- 30
- フォーク
- 6
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
I've been working on a self-hostable Logseq Sync backend, and I was having trouble with the issued STS credentials. The flow looked like:
- Client calls
/get_temp_credentials - Server issues new credentials via STS, scoped to just the
/temp:<region>/<random uuid>bucket prefix - Client generates presigned URLs to
PUTfiles to - Uploads fail with
SignatureDoesNotMatch
But I noticed it wasn't failing all the time! One out of every ten or twenty tries would succeed, indicating it wasn't some complete misconfiguration. There are many, many threads about the SignatureDoesNotMatch issue (here's a big one), some are user error, but many seemed to be resolved by regenerating credentials with no /, +, or = in the secret, but I tried that, and the same issue happened.
So to continue debugging, I did a few things:
- I wrote a quick tool that uses the credentials I was generating (via
/get_temp_credentials), and generates signed PUT URLs, then uploads files to them. That worked fine, indicating that the credentials aren't the problem - I swapped out the existing, bespoke
s3-presigncrate with durch/rust-s3, to see if that was the issue, you can see the relevant changes here.
And the swap worked, my local hacked up Logseq client can now reliably upload files with the presigned S3 URLs it generates with the short-lived STS credentials:
15:26:32.398 › update remote files[txid=1]: ["journals/2023_11_25.md", "pages/This is a test.md"]
15:26:32.626 › upload progress: 100% 360/360 journals/2023_11_25.md
15:26:32.627 › upload progress: 100% 304/304 pages/This is a test.md
15:26:33.758 › copy page file to version-files: "journals/2023_11_25.md"
15:26:33.759 › copy page file to version-files: "pages/This is a test.md"
15:26:33.759 › update remote files success, txid=2
So I'm pretty confident the issue is with the s3-presign package. I don't know what the official Logseq Sync server implementation does (likely during STS credential generation?) such that this issue doesn't occur, but it seems there's some edge case that causes it to generate invalid signatures.
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
調査の方向性
カスタムの s3-presign パッケージと、sync/src/sync.rs の 394-425 行付近にある presigning 呼び出しから始めます。cmd/tools/signtest を使って、その署名付き PUT URL と durch/rust-s3 が生成するものを、短期間有効な STS 認証情報を使って比較します。置き換え後の実装と同様に、SignatureDoesNotMatch が発生せずアップロードが安定して成功すれば完了です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- aws, rust
- 領域
- cloud, security
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100