logseq / logseq/rsapi

SignatureDoesNotMatch issue with s3-presign

Abierto
#2 0 comentarios 1 reacción 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Lenguaje dominante
Rust
Estrellas
30
Forks
6
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

I've been working on a self-hostable Logseq Sync backend, and I was having trouble with the issued STS credentials. The flow looked like:

  1. Client calls /get_temp_credentials
  2. Server issues new credentials via STS, scoped to just the /temp:<region>/<random uuid> bucket prefix
  3. Client generates presigned URLs to PUT files to
  4. Uploads fail with SignatureDoesNotMatch

But I noticed it wasn't failing all the time! One out of every ten or twenty tries would succeed, indicating it wasn't some complete misconfiguration. There are many, many threads about the SignatureDoesNotMatch issue (here's a big one), some are user error, but many seemed to be resolved by regenerating credentials with no /, +, or = in the secret, but I tried that, and the same issue happened.

So to continue debugging, I did a few things:

And the swap worked, my local hacked up Logseq client can now reliably upload files with the presigned S3 URLs it generates with the short-lived STS credentials:

15:26:32.398 › update remote files[txid=1]: ["journals/2023_11_25.md", "pages/This is a test.md"]
15:26:32.626 › upload progress: 100% 360/360 journals/2023_11_25.md
15:26:32.627 › upload progress: 100% 304/304 pages/This is a test.md
15:26:33.758 › copy page file to version-files: "journals/2023_11_25.md"
15:26:33.759 › copy page file to version-files: "pages/This is a test.md"
15:26:33.759 › update remote files success, txid=2

2023-11-25_15-26-36

So I'm pretty confident the issue is with the s3-presign package. I don't know what the official Logseq Sync server implementation does (likely during STS credential generation?) such that this issue doesn't occur, but it seems there's some edge case that causes it to generate invalid signatures.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Comienza con el paquete personalizado s3-presign y la llamada de presigning en sync/src/sync.rs, alrededor de las líneas 394-425. Usa cmd/tools/signtest para comparar sus URLs PUT firmadas con las producidas por durch/rust-s3, utilizando las credenciales STS de corta duración. Se considera terminado cuando las cargas se realizan de forma fiable sin SignatureDoesNotMatch, como ocurre con la implementación de reemplazo.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
aws, rust
Área
cloud, security
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.