databrickslabs / databrickslabs/ontos
[Feature]: Fix delivery pipeline — DELETE handling + one-per-record migration
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 212
- Forks
- 71
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 43
Description
Parent PRD
#254
What to build
End-to-end fix for the indirect delivery pipeline covering three concerns:
1. File model migration: Switch DataDomainFileModel, RoleFileModel, and TagFileModel from ONE_FILE_PER_RECORD=False (single shared file) to ONE_FILE_PER_RECORD=True (individual files per record). Each gets a proper get_filename(entity) returning a sanitized name like hr-domain.yaml, admin-role.yaml, classification-namespace.yaml. The serialize_all code path in FileModel base is retained but unused.
2. DELETE handling in _deliver_indirect: Detect *_DELETE change types and call git_service.delete_file() with the correct filename and subdirectory for the entity type. For unknown entity types (no file model), delete the raw change file.
3. Manager DELETE wiring: Add _queue_delivery(entity, DeliveryChangeType.*_DELETE, ...) calls in the delete methods of all 5 existing delivery-enabled managers (DataProducts, DataContracts, Datasets, DataDomains, Tags). Refactor SettingsManager inline delivery logic for AppRoles (~lines 1770-1824) to inherit DeliveryMixin instead.
4. Unit tests: Test CREATE writes a YAML file, UPDATE overwrites the correct file, DELETE removes the correct file, unknown entity type falls back to raw change data — all via mock GitService.
Acceptance criteria
-
DataDomainFileModel,RoleFileModel,TagFileModeluseONE_FILE_PER_RECORD = Truewith per-entity filenames -
_deliver_indirecthandles*_DELETEchange types by callingdelete_file() - All 5 existing delivery-enabled managers call
_queue_deliveryon delete operations -
SettingsManagerusesDeliveryMixinfor AppRole delivery (inline logic removed) - Unit tests cover CREATE, UPDATE, DELETE, and unknown-entity-type fallback paths
Blocked by
None — can start immediately.
User stories addressed
- User story 4 (deleted contracts removed from Git)
- User story 11 (domain changes don't clobber each other)
- User story 14 (product deletions reflected in Git)
- User story 15 (role and tag changes don't clobber)
- User story 18 (K8s-style resource format maintained)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing FileModel and the DataDomainFileModel, RoleFileModel, and TagFileModel implementations, then follow _deliver_indirect and the DeliveryMixin queue calls. Review delete methods in the five named managers and the SettingsManager AppRoles logic around lines 1770-1824. Done means per-record filenames, correct DELETE behavior and fallback handling, migrated manager wiring, and mock GitService tests for CREATE, UPDATE, DELETE, and unknown entities.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100