deso-protocol / deso-protocol/backend

Bug in GetPostsStateless API Request

Open
#356 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
139
Forks
84
PR merge metrics
No merged PRs in 30d

Description

GetPostsStateless is not working properly when iterated. For example, look at the following nginx requests below. Notice the size of the payloads all are within a somewhat normal range, until the last request. Notice the size of the payload returned on the last request (10312051). That last request is the equivalent of about 1900 posts.

Basically, every time I page to the 5th or 6 "page" of results, it refuses to respect the *NumToFetch* param and simply returns thousands of posts 🤷🏾‍♂️!

I verified the issue is specifically related to the value passed into PostHashHex. If I start a call with a PostHashHex say 300 items deep in the global, the error will happen on the 1st call.

`nginx | 172.31.14.51 - - [23/May/2022:12:44:10 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 118044 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:10 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 145907 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:15 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 135363 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:20 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 144301 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:28 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 150703 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:31 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 119172 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:34 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 107212 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:39 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 4460 "-" "Mozilla/5.0
nginx | 172.31.14.51 - - [23/May/2022:12:44:45 +0000] "POST /api/v0/get-posts-stateless HTTP/1.1" 200 **10312051** "-" "Mozilla/5.0

Also, you can see the same issue from a different angle. The screenshot below is the browser console. Notice again the size of the last response payload @ 10.3mb!

![Screenshot 2022-05-23 054723](https://user-images.githubusercontent.com/3606705/169822605-33813d28-8b62-4683-8d0b-af442d2596db.png)

The object below is the request payload for every call. Again, depending on the PostHashHex passed in, it does not respect the *NumToFetch* value.

```j
{PostHashHex: "8425dd56335283ec0a8ff17a7d5b77aee423d19e703881910327075975ed3f47",…}
AddGlobalFeedBool: true
FetchSubcomments: false
GetPostsByDESO: false
GetPostsForFollowFeed: false
GetPostsForGlobalWhitelist: true
MediaRequired: false
NumToFetch: 25
OrderBy: ""
PostContent: ""
PostHashHex: "**8425dd56335283ec0a8ff17a7d5b77aee423d19e703881910327075975ed3f47**"
PostsByDESOMinutesLookback: 0
ReaderPublicKeyBase58Check: "BC1YLgQMDskGR7m7V3uRqh62wFCZsPgVWH5BFHijArSRLJyPCVo9AuR"
StartTstampSecs: null```

Any ideas what's causing this? This seems pretty important, as we need this working to enable a better scrolling experience for users.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the HTTP handler for POST /api/v0/get-posts-stateless and how it processes PostHashHex and NumToFetch. Reproduce the request with NumToFetch set to 25 and a PostHashHex deep in the global feed; done means iteration consistently returns no more than the requested number of posts.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.