lbryio / lbryio/reflector.go

Blobs upload proposal

Open
#48 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
15
Forks
13
PR merge metrics
No merged PRs in 30d

Description

1) send sdblob in this format:
```
type request struct {
hash string
sdblob []byte
}
```
2) server checks the blocklist to see if it wants the stream
3) server checks if the sdblob is already in our system
3.1) if the sd blob exists, respond with a list of blobs it needs
3.2) if the sd blob doesn't exist, store the sd blob, save the blobs in the DB with is_stored=0, respond with a list of the blobs it needs (all of them)
4) client sends one blob at a time with the following format
```
type request struct {
hash string
blob []byte
size int
}
```
5) server checks that the uploaded blob has the correct hash and size, saves the blob to S3, sets is_stored=1 in the database, responds with a success
6) optionally we should add an endpoint that accepts a list of sdblob hashes and returns whether the sdblob is needed or not and if the sdblob is already stored, a list of blobs that are missing
the server thus has to implement the following endpoints:
`/sdblob POST`
accepts:
```
type request struct {
hash string
sdblob []byte
}
```
returns:
```
type response struct{
success bool
error *error
type data struct {
required_blobs []string
}
}
```
`/blob POST`
accepts:
```
type request struct {
hash string
blob []byte
size int
}
```
returns:
```
type response struct{
success bool
error *error
}
```
`/sdblobs POST`
accepts:
```
type request struct {
hashes []string
}
```
returns:
```
type response struct{
success bool
error *error
type data struct {
required_sdblobs []string
required_blobs []struct{
sdhash string
missing_blobs []string
}
}
}
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the server entry points for the /sdblob, /blob, and /sdblobs POST endpoints, then inspect the blocklist, database, and S3 integrations. Done means the three request and response flows implement the listed checks, blob-state updates, missing-blob responses, and hash and size validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
api, backend, cloud, database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.