Metadata Service (and docs service?)

At the moment, metadata for an adaptor (salesforce, dhis2) is generated by shelling out to the CLI.

This needs refactoring into some kind of long-lived service which does the same job without using the CLI.

I don't know if this is a second Javascript server which is started up with Lightning, or if there's a more effective way to call Javascript from elixirland. A JS server feels like a good and also bad solution.

Some considerations:

• The actual metadata generating logic lives in the adaptor, as JS code. The adaptor provides a metadata() function which returns JSON. This stuff has to live in JS land.
• All the CLI does right now is load and call this function, write the results to the CLI's repo, and add a little bit of extra metadata to the result (a created timestamp). Most of that wrapping stuff can be done in Elixir - but of course you still need to call a JS function somehow.

Related

We have a similar problem with adaptor docs. At the moment, we build the adaptor doc JSON in the client by pulling down a) the typescript engine and b) all the d.ts files associated with the adaptor.

The heavy lifting of generating docs metadata comes from JS code in kit/describe-package. This uses the Typescript engine to parse definition files and extract the information we need.

This needs supporting with a backend service, with caching, which will greatly improve the speed of docs rendering.

JS server

The requirement for two different JS-focused services makes a JS server more tempting.

Is this a responsibility that could be served by the worker? This is a very different sort of relationship to the attempts queue, though. Its a push from Lightning. At the moment Lightning doesn't know (or care) where the Worker is. For this service it would have to call out to an endpoint.

Start by reading the adaptor metadata() function and the CLI flow that calls it, then inspect kit/describe-package and its TypeScript engine and d.ts inputs. Decide how the backend should expose both metadata operations and adaptor docs generation, including caching. Done means Lightning no longer shells out to the CLI or builds docs in the client.