databrickslabs / databrickslabs/ontos
[Feature]: Onboarding Profiling: Tracer bullet — schema + first-N sample end-to-end
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 212
- Forks
- 71
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 43
Description
Parent PRD
#172
What to build
The first end-to-end slice of the onboarding data profiling feature. This establishes the full backend-to-frontend stack: new Pydantic models, a ProfileQueryBuilder utility (pure SQL generation), a ProfilingManager controller, FastAPI routes, and a TableProfiler React component with Schema and Sample tabs.
Backend:
- New
profiling.pyPydantic models:SchemaInfo,ColumnInfo,SampleRequest,SampleResponse,SampleModeenum,TableCount - New
ProfileQueryBuilder: pure functions that take FQN + parameters → SQL strings. Start withbuild_sample_query(fqn, mode=first, limit)andbuild_count_query(fqn). - New
ProfilingManager:get_schema(fqn)via UC tables API,get_count(fqn)via UC metadata or SQL fallback,get_sample(fqn, mode=first, limit)via Statement Execution API. Uses OBO workspace client. Stateless. - New
profiling_routes.pywith three endpoints:GET /api/profiling/{fqn:path}/schemaGET /api/profiling/{fqn:path}/countGET /api/profiling/{fqn:path}/sample?mode=first&limit=100
- Startup wiring: instantiate
ProfilingManager, add toapp.state, add dependency getter - Hard guardrails: max limit of 10,000 rows, query timeout
Frontend:
- New
TableProfilercomponent: inline panel with two tabs (Schema, Sample Data) - Props:
fqn,assetType("table" | "view"), optionalinitialSchema,onClose,defaultMode,defaultLimit - Schema tab loads instantly from
/schemaendpoint; Sample tab fetches on mount with first-N mode - Results cached in React state within the component (no re-fetch on tab switch)
- Error states: no warehouse configured → schema works, sample disabled with banner; permission denied → friendly message; table not found → clear message
Tests:
- Unit tests for
ProfileQueryBuilder.build_sample_queryandbuild_count_query - Unit tests for
ProfilingManagerwith mockedWorkspaceClient(schema retrieval, error handling) - Integration test for routes with
TestClient
See PRD #172 sections "API Design", "Error Handling", and "Testing Decisions" for full details.
Acceptance criteria
-
GET /api/profiling/{fqn}/schemareturns column names, types, and comments from UC metadata -
GET /api/profiling/{fqn}/countreturns row count -
GET /api/profiling/{fqn}/sample?mode=first&limit=100returns first 100 rows as JSON -
limitparameter is capped at 10,000 - Frontend
TableProfilercomponent renders with Schema and Sample tabs - Schema tab loads without warehouse; Sample tab shows banner when no warehouse configured
- Permission denied and table-not-found errors show friendly messages (non-fatal)
-
ProfileQueryBuilderunit tests cover first-N sample and count SQL generation -
ProfilingManagerunit tests cover schema retrieval and error handling with mocked workspace client
Blocked by
None — can start immediately.
User stories addressed
- User story 1
- User story 9
- User story 12
- User story 24
- User story 25
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with PRD #172, especially its API Design, Error Handling, and Testing Decisions sections, then map the backend work across profiling.py, ProfileQueryBuilder, ProfilingManager, and profiling_routes.py. Review the TableProfiler entry point and the named unit and TestClient integration tests; done means the listed schema, count, sample, guardrail, error-state, and frontend acceptance criteria are covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, python, react, sql
- Domain
- api, databases, full-stack
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100