nspcc-dev / nspcc-dev/neofs-node

Do not unmarshal API requests

Open
#4,078 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement I4 neofs-storage performance S3 U4
Dominant language
Go
Stars
38
Forks
51
Avg merge
1d 20h
Merged PRs (30d)
38

Description

Is your feature request related to a problem? Please describe.

efficiency of gRPC deserialization leaves much to be desired. We've optimized handling of response messages and gained visible performance impact

any request is a protobuf message. When server accepts API call, it:

  1. allocates buffer (each one is up to 16K, multiple buffers can be needed)
  2. reads network data into it
  3. unmarshals message and passes it into registered handler like https://github.com/nspcc-dev/neofs-node/blob/6180c090af6c0e73240de230b58d946d8e5f92be/pkg/services/object/server.go#L664

so, we can make similar optimizations made for responses

Describe the solution you'd like

  1. accept binary buffer list in handler
  2. iterate over its fields checking whether we face a valid protobuf and making insta app checks (CID/OID, header, tokens, etc.)
  3. memorize data needed for further processing

primitive data like CID/OID can be caught and left on stack. Structures like bearer token or object header can be transmitted in binary format. For example, we could save object header as BLOB. Or scan bearer token rules w/o allocs. Or, if new messages with same fields will be needed, we can make simple memcpy w/o extra decode-encode round

the code will definitely become more complicated. But we'll get rid of many allocations and increase RPS

it's easier to start with simpler queries like HEAD

Additional context

https://github.com/nspcc-dev/neofs-sdk-go/pull/818

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at pkg/services/object/server.go around line 664 and review the request path alongside the response optimizations referenced in neofs-sdk-go PR #818. Begin with simpler HEAD queries, then assess binary-buffer handling and validation for the listed fields. Done means fewer allocations and improved RPS without losing request validation or handler behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, grpc
Domain
backend-api-design, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.