microsoft / microsoft/FluidFramework

Snapshot, summary, Tree entries: Need standardization and reduction of concepts.

Open
#4,683 11 comments 1 reaction 1 assignee View on GitHub

Nobody has claimed this yet.

ado api design-required epic focus kr refactor
Dominant language
TypeScript
Stars
4.9k
Forks
586
Avg merge
1d 15h
Merged PRs (30d)
146

Description

Our File I/O types are very confusing - it's very hard to follow code on where we use certain types and why:

ISnapshotTree - used when loading from storage, could have name collisions between blobs & trees.
ITree / ITreeEntry - entries, could have name collisions; used when attaching DDS / data store, also used on legacy path
git.ITree - somewhat similar shape as above, mostly used in R11s, but leaks to various helper functions
ISummaryTree - summarization process, no collisions, supports symbolic reuse of content (through handle + path used as a reference)

We have a bunch of helper functions to convert from one type to another:
convertSnapshotTreeToSummaryTree: ISnapshotTree -> ISummaryTreeWithStats
convertToSummaryTree: ITree -> ISummarizeResult
convertSummaryTreeToITree : ISummaryTree -> ITree
buildSnapshotTree: ITree -> ISnapshotTree

While we can't completely remove all these types from runtime (as some are tied to file format), I believe we could reduce it to a minimum, by always converting any time to summary type and all the code working only with summaries.
This would substantially reduce mental workload when working with all these layers.

This work can be done incrementally, but adding more converter functions to support any to any conversions, and thus allowing us to address layer by layer, keeping system functional during this process.

Somewhat good example why not acting causes more types used even on same path:
We summarize using summaries. But because we load using snapshots, IRuntime.stop() method uses snapshot() path to get snapshot of container, forcing us to have both types of serialization. Ideally both save & load would use same format, but if they can't, we should serialize only in one format, and use helper functions (here - at Container level) to convert to write format.

  • #8460
Completed
  • #3929
  • #8652
  • #8595
Independent cleanup tasks

These can be done at any time.

  • #4728
  • #4691
  • #4434
  • #9003
Eliminate ITree usages

This gets us close to being able to remove ITree from runtime. There might still be some usages in tests and utils.

  • #8997
  • #9250
  • #9227
  • #9251

We could consider renaming since "snapshot" and "summary" indicate nothing about the directional distinction.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.