imazen / imazen/imageflow-server
RFC: Transmission of binary data with JSON
- Dominant language
- C#
- Stars
- 316
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
There's already a good writeup on this topic: http://blog.marcinbudny.com/2014/02/sending-binary-data-along-with-rest-api.html
We need to send binary image data, often in excess of 10MB, between the client and server. JSON Base64 increases byte count by 33%, but also has decode/encode overhead.
Key questions for *your* platform/language:
1. How efficient is your JSON implementation and decoding and encoding enormous base64 strings? With Rust<->Rust we could may see 40ms overhead for 4 codings of a 10MB image.
2. Does your HTTP client support sending multipart requests and parsing multipart responses?
Please speak up if you lack both multipart support and a fast JSON codec.
## JSON implementations
... are free to [set limits on the maximum size of strings or the message itself](http://stackoverflow.com/a/30739035).
Vary widely in speed: https://www.reddit.com/r/programming/comments/3pojrz/the_fastest_json_parser_in_the_world/
Are ubiquitous and usually optimized. Benchmarks of huge string parsing and base64 decoding appreciated.
## BSON
BSON has a native binary datatype, but implementations tend to be less optimized than their JSON counterparts. Benchmarks of huge binary blobs appreciated.
## HTTP libraries
Benchmarks of multipart parsing overhead on your platform would be appreciated.
## When it doesn't matter so much
* We can specify URLs instead of sending input blobs, and offload that fetch (and potentially, have some kind of caching).
* We can ask the server to just send the first image result instead of using multipart. We still have to handle failure JSON, though.
## Non-HTTP interactions
* libimageflow users can send pointers to buffers.
* imageflow_tool can read/write to the local filesystem.
Contributor guide
No contributing guide indexed for this repository
Research direction
No files or tests are named. Start with the linked writeup, then gather the requested benchmarks for JSON/Base64 and multipart handling on the repository's C# platform; done means documenting a selected transport approach and its trade-offs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, json
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100