RFC: Self-hosting - Use generated protocol models and metamodels
- Dominant language
- TypeScript
- Stars
- 111
- Forks
- 149
- Avg merge
- 9h 39m
- Merged PRs (30d)
- 63
Description
## 💬 Request for Comments
We are a modelling tool, in fact, our protocols and metamodels are models we could build in Pure as well, and worse yet, we are not auto-generating these models but repeating their definition in 3 code bases `legend-studio`, `legend-engine` and `legend-pure`. We should model these in Pure and code-gen their definition into Typescript for consumption in Studio codebase.
### Simplify metamodel
We need to simplify metamodels so that these can be auto-generated from models in Pure. After this, we're sure that the structure of metamodel more consistent and easier for code-generation
- [x] Move the `lambdaId` to stores - https://github.com/finos/legend-studio/pull/227
- [x] Move `PackageableElementSelectOption` back into `stores`, don't leave those in `graph` and `metamodels`
- [x] Remove rendering logic like `label`, e.g. `MappingElementLabel` - https://github.com/finos/legend-studio/pull/1068
- [x] For fields that we added to support better graph navigation (i.e. `owner`, `parentClass`, etc.`) that are not in the actual metamodel, we should prefix them with `\_`, such as `\_owner` - https://github.com/finos/legend-studio/pull/1153
- [x] Move helpers logic into corresponding classes `MappingHelpers`, `PackageableElementHelpers` like in `finos/legend-pure` - https://github.com/finos/legend-studio/pull/1159
- [x] ~~`IMPORTANT` Add field integrity verification to `hashCode` (basically check if properties annotated with ! are actually non-null)~~ - Detailed in https://github.com/finos/legend-studio/issues/288 but no longer deem needed
- [x] Consider to remove unnecessary interfaces (these make it hard for tools like VSCode to know where the interface method being called):
- [x] `Stubable` interface -> Have a factory to create stub and check for stub maybe? -> move the logic out for stubable to a factory or some helper methods to initialize new instances - https://github.com/finos/legend-studio/pull/1159
- [x] ~~`Hashable` interface, we don't need models to implement this interface anymore.~~ - see below
- [x] ~~Move `validation` logic to `top-down` as well~~ - we will most likely do this later in #1168
- [x] For certain `@computed` cases (e.g. `allSuperClasses`, `allSubClassses`) we might not need that as `computed` value at all - https://github.com/finos/legend-studio/pull/1159
- [x] `MAYBE` Turn ALL metadata fields (fields like `Property`'s `owner`, `uuid` etc.) to `readonly` and leave the access modifier as `public` - ONLY during initialization do we allow to change these `readonly` property using the `Writable` [improved mapped type modifiers](https://stackoverflow.com/questions/46634876/how-can-i-change-a-readonly-property-in-typescript) trick in Typescript, write a good doc there to explain why this `abuse` should only be treated as a hack used ONLY during graph initialization. Or we can simply use `Object.assign` syntax... - https://github.com/finos/legend-studio/pull/1153
```typescript
type Writable = { -readonly [K in keyof T]: T[K] };
```
> NOTE: there are fields that the UI does not and perhaps should not allow changing like `class` to a `ClassView`, but that's an UI decision, this field is not metadata, and it should not be made `readonly`
- [ ] Remove all constructor parameters. Then remove all initialization logic from `constructor`, in fact, we could `remove/make private` the constructor all together and replace them by `static` creator methods
- [ ] Do not initialize any value, as these should be taken care of by the static creator method
- [ ] Use `!` and `?` in the property definition
- [ ] Remove constructor
- [ ] Move hashing logic out of metamodels:
- [ ] After we move `hashing` and `validation` logic, revisit spots where we should use `isStubbed_...` method and use them, right now to avoid bad dependencies between models and the helpers method, we don't do this
- [ ] Move remaining code-logic in `DSL Diagram classes` out to helpers.
- [ ] Consider moving logic in `Packageable Element` out
- [ ] Use the following technique to move hashing out. Remember that this is useful for derivations, which are the hardest things left to move. Hashing is one example of derivations which is useful to us and we should keep around. Also, if we don't do hashing at protocol level, I think doing `hashing` in the reactive manner is still the best, but we will see
```js
// ModelExtension.ts
declare module './models/metamodels/pure/packageableElements/PackageableElement' {
interface PackageableElement {
get path(): string;
get hashCode(): string;
}
}
...
export {}; // export this so `Typescript` --isolatedModule check does not complain
// --------------------------------------------------------------
// Some file that we call once when we load the app
export const decorateMetamodel = (): void => {
// e.g. Constraint
Object.defineProperty(Constraint.prototype, 'hashCode', {
get: function (this: Constraint) {
return hashArray([
HASH_STRUCTURE.CONSTRAINT,
this.name,
]);
}
});
};
```
### Other considerations
- [ ] `CONSIDER` We probably would need to take care of visitors as part of code generation. `JS` does not support overloading like `Java` so we can't really have 2 `accept()` methods, we would need to have them differentiated, e.g. `accept_SetImplementationVisitor()` and `accept_PropertyMappingVisitor()` for embedded property mappings.
- [ ] `CONSIDER` How do we support Typescript `union type`? Maybe we can use `Any` in Pure and then use a tagged-value to annotate union type?
#### Class/Interface swap:
Since `Pure` support multiple inheritance, we might need to do what `legend-pure` does by turning metamodels into `interfaces` to support multiple-inheritance (with `C3 linearization` support?), something like the following
- [ ] Create `CoreInstance` interface and `ConcreteCoreInstance` class:
```typescript
interface CoreInstance {
/**
* Can be used to uniquely identify the instance in a collection (used for UI majorly)
*/
_UUID?: string;
}
class ConcreteCoreInstance implements CoreInstance {
_UUID?: string;
}
```
- [ ] We should turn each class into `_Impl` (that extends `ConcreteCoreInstance`) and have everything as interface that extends `CoreInstance`: i.e. `Class` vs `ClassImpl`.
- [ ] `MAYBE` We should have **optional** UUID to each `CoreInstance`, add a method to allow setting this UUID. **NOTE that if we make this non-optional, we're adding a lot of data to the graph**
- [ ] At this point, we lose the ability to do `instanceof`, to recover, for each interface, we can do the following:
```typescript
// Class.ts
export interface Class {
...
}
export const Class_InheritanceSignature = 'Class';
// Class_Impl.ts
class Class_Impl implements Class {
...
}
Class_Impl.inheritanceSignature = Class_InheritanceSignature;
Class_Impl.prototype.superTypes = [PackageableElement_InheritanceSignature, CoreInstance_InheritanceSignature];
const instanceOf = (val: unknown, inheritanceSignature: string): val is T => val.protoype.superTypes.includes(inheritanceSignature);
```
```typescript
interface A {
type: "1";
}
interface B {
type: "2";
}
interface C extends A {}
class D implements C {
type: "1" = "1";
}
class E implements B {
type: "2" = "2";
}
const instanceOfA = (val: Record): val is A => {
return "type" in val && val.type === "1";
};
const invD = (d: D): void => {
console.log(d.type);
};
// NOTE: this line is perfectly fine in Javascript, but during compile time, Typescript will complain
invD(new E()); // Argument of type 'E' is not assignable to parameter of type 'D'
```
Contributor guide
Research direction
This RFC covers a broad self-hosting effort across legend-studio, legend-engine, and legend-pure, with unchecked work on constructors, hashing, visitors, and interface generation. Start by reviewing the unchecked sections and the ModelExtension.ts example; the issue does not define a single entry point, scoped deliverable, or completion test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- developer-experience, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100