NVIDIA-NeMo / NVIDIA-NeMo/DataDesignerPlugins
Extend ddp new to scaffold all DataDesigner plugin types
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
Parent epic: #15
Depends on: #21
Why
The runtime plugin system supports column generators, seed readers, and processors. The current scaffold emits only a column-generator package, so DDPlugins is not yet a complete reference tap authoring workflow.
CLI contract
Extend ddp new to:
uv run ddp new my-plugin --type column-generator
uv run ddp new my-plugin --type seed-reader
uv run ddp new my-plugin --type processor
Backwards compatibility:
uv run ddp new my-plugin
must continue to create a column-generator plugin and the help text must say that this is the default.
Generated column-generator template
- Config class subclasses
data_designer.config.base.SingleColumnConfig. - Discriminator field is
column_type: Literal["<slug>"] = "<slug>". - Implementation class subclasses
ColumnGeneratorFullColumn[ConfigClass]. - Plugin object uses
PluginType.COLUMN_GENERATOR. - Test imports
pluginand callsassert_valid_plugin(plugin).
Generated seed-reader template
- Config class subclasses
data_designer.config.seed_source.FileSystemSeedSource. - Discriminator field is
seed_type: Literal["<slug>"] = "<slug>". - Implementation class subclasses
data_designer.engine.resources.seed_reader.FileSystemSeedReader[ConfigClass]. - Generated implementation should be a minimal placeholder that imports cleanly and clearly marks
get_dataset_uri/ DuckDB connection behavior as TODOs rather than pretending to be production-ready. - Plugin object uses
PluginType.SEED_READER. - Test imports
pluginand callsassert_valid_plugin(plugin).
Generated processor template
- Config class subclasses
data_designer.config.base.ProcessorConfig. - Discriminator field is
processor_type: Literal["<slug>"] = "<slug>". - Implementation class subclasses
data_designer.engine.processing.processors.base.Processor[ConfigClass]. - Generated implementation overrides
process_after_batchand returns data unchanged with a TODO. - Plugin object uses
PluginType.PROCESSOR. - Test imports
pluginand callsassert_valid_plugin(plugin)if DataDesigner validates processor plugins through that helper; otherwise the test should cover public import and plugin object construction while documenting the DataDesigner validation gap.
Shared generated behavior
- Use tap config from #21 for package prefix, author, repository URL, docs URL examples, and
data-designerdependency. - Use absolute imports only.
- Use Python 3.10+ annotations.
- Generate README and docs text that says “plugin type” or “entry point,” not “column type,” unless the generated plugin is specifically a column generator.
- Add scaffold golden tests for file tree and important file contents for all three plugin types.
- Add CLI tests for valid types, invalid type errors, and default behavior.
Acceptance criteria
ddp new my-plugin --type column-generatorcreates a valid column-generator plugin.ddp new my-plugin --type seed-readercreates a valid seed-reader plugin.ddp new my-plugin --type processorcreates a valid processor plugin or a documented processor-validation-limited scaffold if DataDesigner lacks full validation.uv run pytest devtools/ddp/tests/test_cli.py devtools/ddp/tests/test_scaffold.py -qcovers all variants.- Generated docs and plugin cards do not label seed readers or processors as column types.
Dependencies
- Depends on: #21.
- Related DataDesigner dependency marker:
DD-CORE-PROCESSOR-PLUGIN-VALIDATION. - Blocks: external tap authoring workflow docs in #25.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with dependency #21 and the ddp new entry point, then inspect devtools/ddp/tests/test_cli.py and devtools/ddp/tests/test_scaffold.py. Cover column-generator, seed-reader, and processor scaffolds, including valid and invalid types and the default behavior. Done means the generated trees, contents, docs, and plugin tests are covered by uv run pytest devtools/ddp/tests/test_cli.py devtools/ddp/tests/test_scaffold.py -q.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100