smart-data-lake / smart-data-lake/sdl-visualization

Read column descriptions from the schema, remove _columnDescriptions

Open
#129 0 comments 0 reactions 1 assignee View on GitHub

@zzeekk is already working on this.

Since Sep 15, 2026.

Dominant language
TypeScript
Stars
4
Forks
1
Avg merge
2h 58m
Merged PRs (30d)
6

Description

Describe the cleanup

SDLB no longer exports _columnDescriptions in exportedConfig.json. Column descriptions written with
@column in the DataObject markdown files are now applied as column comments — to the data catalog and to
the exported schema JSON — by DataObjectSchemaExporter. The UI should read them from the schema like any
other comment, and drop the separate _columnDescriptions path.

This is the UI side of smart-data-lake/smart-data-lake#1121, where table and column metadata moved out of the
SDLB run into a deployment step.

What changed on the SDLB side

  • ConfigJsonExporter.enrichColumnDescription was removed, so dataObjects.<id>._columnDescriptions is gone
    from the exported config. --descriptionPath / --uploadDescriptions still exist and still upload the
    markdown files themselves.
  • The parsing moved to a shared ColumnDescriptionParser, used by DataObjectSchemaExporter.
  • DataObjectSchemaExporter --mode apply merges the @column descriptions over the comments in the
    exported schema (markdown wins) and writes the result to the catalog. The schema JSON that the UI reads
    therefore already contains the final, consolidated comment in each field's comment attribute.

Places to change

  • src/components/ConfigExplorer/ElementDetails.tsx:118 — stop passing
    columnDescriptions={configObj?._columnDescriptions} to SchemaTab.
  • src/components/ConfigExplorer/SchemaTab.tsx — drop the columnDescriptions prop (declared at :54, read at
    :93, :104, :114, :130) and use column.comment alone.
    Note numberSchemaTree (:130-133) currently concatenates the markdown description and the schema comment
    when both exist. That merge now happens in SDLB, with the markdown description overriding the schema comment,
    so the concatenation should simply go away rather than be reimplemented.
  • src/components/ConfigExplorer/ConfigurationAccordions.tsx:98_columnDescriptions can be removed from
    propsToIgnore, since the attribute no longer appears in the config.

Compatibility

_columnDescriptions was an optional attribute, so the UI just needs to stop relying on it. Configs exported by
an older SDLB will still contain it; ignoring it is fine.

One behavioural note

SDLB maps a markdown column path to the SQL column path, dropping array markers: b.[].b1 becomes b.b1, which
is the struct field inside the array — the common case, and it works. The UI additionally supports descriptions
attached to the array/map pseudo-rows themselves (-element-, -key-, -value-, addressed as a.[], a.key,
a.value). Those pseudo-nodes have no comment of their own in the schema JSON, so a description written
specifically for them is no longer displayed. Worth confirming whether anyone relies on that.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.