How are Buffers converted to string internally?
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 17.6k
- Forks
- 2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 40
Description
I apologize in advance for asking the question here - I've originally posted my question on stackoverflow, but after 10 days I still didn't manage to get the answer that I was looking for... I thought I might try my luck here 😄. If it's not ok to ask the question here, let me know and I'll remove it...
I'm not having any issues with the library, but I'm trying to figure out how Buffer is being handled in node-redis library. I have my nodejs script that fetches some data from database, serializes and gzips the result and then stores that result in redis. Later I read this value in php script, so I need to match what php script expects
In node gzip library returns Buffer and what I'm wondering is how does node-redis convert this Buffer to string internally?
When I pass Buffer (from gzip) to redis, it will be saved in the format that my php script can read and decode properly
await client.setex('compressed', 3600, compressed);
Value in redis (result is from redis desktop manager terminal)
redis1:0>get "compressed'
"\x1F\x8B\x08\x00\x00\x00\x00\x00\x02\x03\xABVJI,IT\xB2R*I-.Q\xAA\x05\x00\xC0\xE3/\xAC\x0F\x00\x00\x00"
But when I pass Buffer.toString() to redis, php won't be able to read and decode
await client.setex('compressed', 3600, compressed.toString());
Value in redis
redis1:0>get "compressed-toString"
"\x1F\xEF\xBF\xBD\x08\x00\x00\x00\x00\x00\x02\x03\xEF\xBF\xBDVJI,IT\xEF\xBF\xBDR*I-.Q\xEF\xBF\xBD\x05\x00\xEF\xBF\xBD\xEF\xBF\xBD/\xEF\xBF\xBD\x0F\x00\x00\x00"
Now I'm trying to figure out how exactly is node-redis converting this buffer to string internally? Why are these 2 values different? Best that I could find is this stream write, but I'm still not clear on how exactly it works and why are the redis values different...
Any help is appreciated!
Test example
I've created simple example
Environment
-
Node.js Version:
v12.18.3 -
Redis Version:
5.0.5(redis:5.0.5-alpine) -
Platform:
Ubuntu 20.04
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the referenced index.js stream-write code and the linked node-redis-test example, then trace how the client handles a Buffer versus Buffer.toString(). Compare the observed Redis values with the Node.js and Redis versions listed. Done means documenting the conversion behavior and why the two stored values differ.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, redis
- Domain
- backend, database
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100