NVIDIA-NeMo / NVIDIA-NeMo/DataDesignerPlugins
Generate schema v2 catalog fields from ddp sync catalog
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
Parent epic: #15
Depends on: #16, #21
Why
The catalog must remain generated, not hand-edited. Today ddp sync catalog renders schema v1 from package metadata, installed data_designer.plugins entry points, and direct data-designer dependency constraints. It needs to render the concrete schema v2 contract from #16 using the repo-level tap metadata from #21.
Implementation
- Set
CATALOG_SCHEMA_VERSION = 2. - Extend
CatalogEntrywith:source: dict[str, object]docs_url: str
- Generate default NVIDIA entries with this source object because
[tool.ddp.tap].default-source = "pypi":
{"type": "pypi", "package": "<project.name>"}
- Generate
docs.urlas:
f"{docs_base_url.rstrip('/')}/plugins/{normalize_docs_slug(project_name)}/"
where normalize_docs_slug should match the existing plugin docs slug logic.
- Preserve current v1 fields exactly unless #16 says otherwise:
name,plugin_type,description,package,entry_point, andcompatibility. - Keep
compatibility.data_designer.requirement,specifier, andmarker; the PDF example omitsrequirement, but the current field is useful for explanations and should remain. - Add duplicate runtime plugin-name detection before rendering JSON.
- Add source-object validation for the default
pypisource. - Regenerate
catalog/plugins.json. - Update
devtools/ddp/tests/test_catalog.pyexpected output to schema v2. - Add a multi-entry fixture test where two entry points in one package share identical
packageandsourcemetadata.
Expected generated entry
For the template plugin, schema v2 output should be structurally equivalent to:
{
"name": "text-transform",
"plugin_type": "column-generator",
"description": "Template Data Designer plugin — text transform column generator",
"package": {
"name": "data-designer-template",
"version": "0.1.0",
"path": "plugins/data-designer-template"
},
"entry_point": {
"group": "data_designer.plugins",
"name": "text-transform",
"value": "data_designer_template.plugin:plugin"
},
"compatibility": {
"python": {"specifier": ">=3.10"},
"data_designer": {
"requirement": "data-designer>=0.5.7",
"specifier": ">=0.5.7",
"marker": null
}
},
"source": {
"type": "pypi",
"package": "data-designer-template"
},
"docs": {
"url": "https://nvidia-nemo.github.io/DataDesignerPlugins/plugins/data-designer-template/"
}
}
Acceptance criteria
make catalogemitsschema_version: 2.- Every generated plugin entry has
source.type = "pypi",source.package = package.name, and adocs.urlderived from[tool.ddp.tap].docs-base-url. - Existing compatibility fields remain generated from direct versioned
data-designerdependencies. - Duplicate runtime plugin names fail catalog generation.
- Multi-entry packages render as multiple entries sharing one package/source identity.
uv run pytest devtools/ddp/tests/test_catalog.py -qpasses.make check-catalogfails when checked-in catalog output is stale.
Dependencies
- Depends on: #16, #21.
- Blocks: #18, #19, #20, #23, and DataDesigner CLI read-only catalog work.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing catalog generator behind ddp sync catalog and devtools/ddp/tests/test_catalog.py, including the current plugin documentation slug logic. Run uv run pytest devtools/ddp/tests/test_catalog.py -q and inspect catalog/plugins.json; done means schema version 2 output, validated source and docs fields, duplicate-name failure, multi-entry coverage, and passing catalog checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100