Idea: Memcached client in golang with request pipelining

For https://memcached.org/ servers or proxies

This may be out of scope.

Motivation

• Memcached is heavily used in some web applications for caching state or the results of expensive operations (db queries, results of reading data from web services, etc)

• If there are hundreds of workers, it'd reduce cpu and memory resource usage and syscalls if they shared a small finite number of memcache connections (pool)

  (unless there are ~32+ cpu cores, 1 connection may suffice)

• Both golang and php could use the same client, allowing moving method implementations to/from golang (e.g. check a json array for a value, add a value)

  • (or read/write with serializers such as PECL msgpack, etc. https://msgpack.org/ has many implementations. )
  • Though this could go in its own PECL, the ability to share the same client is useful

Benchmarking notes

• Request pipelining does better when there is actual network latency (https://github.com/twitter/twemproxy/#pipelining) (e.g. non-pipelined golang clients would be limited to 1000 requests per second with 0.001 second network latency)
• https://github.com/twitter/twemproxy/ has performance issues when clients pipeline requests. Reduce mbuf size with -m $MBUF_SIZE if this is configured to connect to a twemproxy proxy
• I haven't benchmarked this yet.

Background

It would be extremely useful to have compatibility with the memcached flags int used to indicate the chosen serializer https://github.com/php-memcached-dev/php-memcached/ (source of http://pecl.php.net/memcached )

• Years ago, I implemented a pipelining memcached client, and embedded it in https://github.com/TysonAndre/golemproxy#about for a hackathon (it should work and has unit tests, but I never had time to try it)
• I believe the pool config syntax in that codebase is compatible with https://github.com/twitter/twemproxy/#pipelining

The memcached client this uses is based on https://github.com/bradfitz/gomemcache

• This rewrites parts of it and adds pipelining support to that library.

Alternatives

Use existing memcached or memcache PECL normally

No files, tests, or entry points are named. Start by reviewing the repository scope and the existing gomemcache-based client referenced in the issue, then determine whether pipelining, pooling, and PHP serializer-flag compatibility belong here. The issue needs an agreed scope and acceptance criteria before completion can be judged.