databrickslabs / databrickslabs/ontos

[Feature]: Fix delivery pipeline — DELETE handling + one-per-record migration

Open
#255 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

scope/settings type/feature
Dominant language
Python
Stars
212
Forks
71
Avg merge
4d 10h
Merged PRs (30d)
43

Description

Parent PRD

#254

What to build

End-to-end fix for the indirect delivery pipeline covering three concerns:

1. File model migration: Switch DataDomainFileModel, RoleFileModel, and TagFileModel from ONE_FILE_PER_RECORD=False (single shared file) to ONE_FILE_PER_RECORD=True (individual files per record). Each gets a proper get_filename(entity) returning a sanitized name like hr-domain.yaml, admin-role.yaml, classification-namespace.yaml. The serialize_all code path in FileModel base is retained but unused.

2. DELETE handling in _deliver_indirect: Detect *_DELETE change types and call git_service.delete_file() with the correct filename and subdirectory for the entity type. For unknown entity types (no file model), delete the raw change file.

3. Manager DELETE wiring: Add _queue_delivery(entity, DeliveryChangeType.*_DELETE, ...) calls in the delete methods of all 5 existing delivery-enabled managers (DataProducts, DataContracts, Datasets, DataDomains, Tags). Refactor SettingsManager inline delivery logic for AppRoles (~lines 1770-1824) to inherit DeliveryMixin instead.

4. Unit tests: Test CREATE writes a YAML file, UPDATE overwrites the correct file, DELETE removes the correct file, unknown entity type falls back to raw change data — all via mock GitService.

Acceptance criteria

  • DataDomainFileModel, RoleFileModel, TagFileModel use ONE_FILE_PER_RECORD = True with per-entity filenames
  • _deliver_indirect handles *_DELETE change types by calling delete_file()
  • All 5 existing delivery-enabled managers call _queue_delivery on delete operations
  • SettingsManager uses DeliveryMixin for AppRole delivery (inline logic removed)
  • Unit tests cover CREATE, UPDATE, DELETE, and unknown-entity-type fallback paths

Blocked by

None — can start immediately.

User stories addressed

  • User story 4 (deleted contracts removed from Git)
  • User story 11 (domain changes don't clobber each other)
  • User story 14 (product deletions reflected in Git)
  • User story 15 (role and tag changes don't clobber)
  • User story 18 (K8s-style resource format maintained)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing FileModel and the DataDomainFileModel, RoleFileModel, and TagFileModel implementations, then follow _deliver_indirect and the DeliveryMixin queue calls. Review delete methods in the five named managers and the SettingsManager AppRoles logic around lines 1770-1824. Done means per-record filenames, correct DELETE behavior and fallback handling, migrated manager wiring, and mock GitService tests for CREATE, UPDATE, DELETE, and unknown entities.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.