RocketChat / RocketChat/Rocket.Chat

Can't upload file to Ceph RGW S3 API with SigV4

Open
#32,849 3 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

planned
Dominant language
TypeScript
Stars
46.1k
Forks
13.9k
Avg merge
3d 3h
Merged PRs (30d)
130

Description

Description:

I did a migration of files to S3. I tried setup before that and I hitted an error, written below.

My setup:

  • Ceph 18.2.2 with RGW and S3 API
  • Nginx proxy before RGW endpoints
  • Proxy mode in RC for every type of files
Steps to reproduce:
  1. Setup SigV4 for S3 in RC
  2. Try to upload a file
Expected behavior:

File uploaded

Actual behavior:

File not uploaded / missing

Server Setup Information:
  • Version of Rocket.Chat Server: 6.9.3
  • License Type: Enterprise
  • Number of Users: 100+
  • Operating System: Linux
  • Deployment Method: K8s helm chart
  • Number of Running Instances: 3
  • DB Replicaset Oplog: yes
  • NodeJS Version: ??? (built of official image)
  • MongoDB Version: 5.0
Client Setup Information
  • Desktop App or Browser Version: 4.0.1
  • Operating System: Arch linux
Additional context

N/A

Relevant logs:

Log message shown in Reports

{"level":50,"time":"2024-07-20T12:23:24.859Z","pid":1,"hostname":"rocketchat-rocketchat-6df49df4db-c4xxt","name":"System","err":{"type":"Error","message":"Non-file stream objects are not supported with SigV4","stack":"Error: Non-file stream objects are not supported with SigV4<br>    at Object.computeSha256 (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/util.js:753:23)<br>    at Request.COMPUTE_SHA256 (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/event_listeners.js:205:18)<br>    at Request.callListeners (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:102:18)<br>    at Request.emit (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:78:10)<br>    at Request.emit (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:686:14)<br>    at Request.transition (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:22:10)<br>    at AcceptorStateMachine.runTo (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/state_machine.js:14:12)<br>    at /app/bundle/programs/server/npm/node_modules/aws-sdk/lib/state_machine.js:26:10<br>    at Request.<anonymous> (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:38:9)<br>    at Request.<anonymous> (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:688:12)<br>    at Request.callListeners (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:116:18)<br>    at Request.emit (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:78:10)<br>    at Request.emit (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:686:14)<br>    at Request.transition (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:22:10)<br>    at AcceptorStateMachine.runTo (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/state_machine.js:14:12)<br>    at /app/bundle/programs/server/npm/node_modules/aws-sdk/lib/state_machine.js:26:10<br>    at Request.<anonymous> (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:38:9)<br>    at Request.<anonymous> (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:688:12)<br>    at Request.callListeners (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:116:18)<br>    at callNextListener (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:96:12)<br>    at /app/bundle/programs/server/npm/node_modules/aws-sdk/lib/event_listeners.js:120:11<br>    at finish (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/config.js:396:7)<br>    at /app/bundle/programs/server/npm/node_modules/aws-sdk/lib/config.js:414:9<br>    at Credentials.get (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/credentials.js:127:7)<br>    at getAsyncCredentials (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/config.js:408:24)<br>    at Config.getCredentials (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/config.js:428:9)<br>    at Request.VALIDATE_CREDENTIALS (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/event_listeners.js:111:28)<br>    at Request.callListeners (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:102:18)<br>    at Request.emit (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/sequential_executor.js:78:10)<br>    at Request.emit (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:686:14)<br>    at Request.transition (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:22:10)<br>    at AcceptorStateMachine.runTo (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/state_machine.js:14:12)<br>    at Request.runTo (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:406:15)<br>    at Request.send (/app/bundle/programs/server/npm/node_modules/aws-sdk/lib/request.js:370:10)","code":"Error","time":"2024-07-20T12:23:24.859Z"},"msg":"Non-file stream objects are not supported with SigV4"}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Search the Rocket.Chat upload and storage integration for S3 SigV4 handling and the AWS SDK call implicated by the reported stack trace. Reproduce with Ceph 18.2.2 behind Nginx using the listed Rocket.Chat settings, then verify that file uploads complete successfully and check the relevant tests for coverage of this path.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, node.js, typescript
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.