Query Server Protocol v2

@janl:
> We need to revamp the Query Server. It is hardcoded to an out-of-date version of SpiderMonkey and we are stuck with C-bindings that barely anyone dares to look at, let alone iterate on.
>
>I believe the way forward is re-vamping the query server protocol to use streaming IO instead of blocking batches like we do now, and use JS-native implementation of the JS-side instead of C-bindings.
>
>I’m partial to doing this straight in Node, because there is a ton of support for things we need already, and I believe we’ve solved the isolation issues required for secure MapReduce, but I’m happy to use any other thing as well, if it helps.
>
>Other benefits would be support for emerging JS features that devs will want to use.
>
>And we can have two modes: standalone QS like now, and embedded QS where, say, V8 is compiled into the Erlang VM. Not everybody will want to run this, but it’ll be neat for those who do.

@davisp:
>I'd be quite happy to see this change made as well for all of the reasons listed and more (busy servers with lots of CouchJS processes sucking up RAM come to mind). And I know others would really like to see better integration with other languages (ie, Lua embedded in the VM has been mentioned a number of times).
>
>However I think the protocol is probably the wrong place to look at for this. I'd instead like us to take a look at our "language" field on design docs and see if we can't come up with either an extension or a more abstract field that we can use.
>
>For instance, your specific scenario where we use Node as an external process vs. Node embedded in the Erlang VM is a great case. Both are JavaScript but both are fairly significantly different on the implication.
>
>The first thing that comes to mind is some sort of content type extension where we use some specific names, ie, "javascript+erlv8" or similar. Or perhaps add the ability to tag things with required features maybe? That way an admin could determine whether there's a node server running or that V8 is embedded and most design docs wouldn't care where their code ran.
>
>However, I think once we explore this aspect, then each of our "language" implementations can write their own protocols and do whatever they want. Thus V8 doesn't have to do anything hacky around not reading/writing data to stdio of a port program and new Node query servers could do something fancy dance. And someone could come along and write a strange BF view server that used shared memory to transfer JSON docs in and out of the view if they so desired.
>
>Or maybe I'm talking crazy talk.

@wohali:
> With the discussion to move ChakraCore into the system, do we really want to bother with continuing to support a query server protocol at all? I'm not so sure.

Also see #1334.

Start with the existing Query Server Protocol and the design-document "language" field, then read the discussion in #1334. Compare the current blocking-batch and C-binding assumptions with the proposed streaming, standalone, and embedded modes; the issue is not done until the protocol or language-selection direction is agreed and specified.

Written by the indexing model from the issue text.