databrickslabs / databrickslabs/ontos

[Feature]: Onboarding Profiling: Tracer bullet — schema + first-N sample end-to-end

Open
#175 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

scope/asset-reviews type/feature
Dominant language
Python
Stars
212
Forks
71
Avg merge
4d 10h
Merged PRs (30d)
43

Description

Parent PRD

#172

What to build

The first end-to-end slice of the onboarding data profiling feature. This establishes the full backend-to-frontend stack: new Pydantic models, a ProfileQueryBuilder utility (pure SQL generation), a ProfilingManager controller, FastAPI routes, and a TableProfiler React component with Schema and Sample tabs.

Backend:

  • New profiling.py Pydantic models: SchemaInfo, ColumnInfo, SampleRequest, SampleResponse, SampleMode enum, TableCount
  • New ProfileQueryBuilder: pure functions that take FQN + parameters → SQL strings. Start with build_sample_query(fqn, mode=first, limit) and build_count_query(fqn).
  • New ProfilingManager: get_schema(fqn) via UC tables API, get_count(fqn) via UC metadata or SQL fallback, get_sample(fqn, mode=first, limit) via Statement Execution API. Uses OBO workspace client. Stateless.
  • New profiling_routes.py with three endpoints:
    • GET /api/profiling/{fqn:path}/schema
    • GET /api/profiling/{fqn:path}/count
    • GET /api/profiling/{fqn:path}/sample?mode=first&limit=100
  • Startup wiring: instantiate ProfilingManager, add to app.state, add dependency getter
  • Hard guardrails: max limit of 10,000 rows, query timeout

Frontend:

  • New TableProfiler component: inline panel with two tabs (Schema, Sample Data)
  • Props: fqn, assetType ("table" | "view"), optional initialSchema, onClose, defaultMode, defaultLimit
  • Schema tab loads instantly from /schema endpoint; Sample tab fetches on mount with first-N mode
  • Results cached in React state within the component (no re-fetch on tab switch)
  • Error states: no warehouse configured → schema works, sample disabled with banner; permission denied → friendly message; table not found → clear message

Tests:

  • Unit tests for ProfileQueryBuilder.build_sample_query and build_count_query
  • Unit tests for ProfilingManager with mocked WorkspaceClient (schema retrieval, error handling)
  • Integration test for routes with TestClient

See PRD #172 sections "API Design", "Error Handling", and "Testing Decisions" for full details.

Acceptance criteria

  • GET /api/profiling/{fqn}/schema returns column names, types, and comments from UC metadata
  • GET /api/profiling/{fqn}/count returns row count
  • GET /api/profiling/{fqn}/sample?mode=first&limit=100 returns first 100 rows as JSON
  • limit parameter is capped at 10,000
  • Frontend TableProfiler component renders with Schema and Sample tabs
  • Schema tab loads without warehouse; Sample tab shows banner when no warehouse configured
  • Permission denied and table-not-found errors show friendly messages (non-fatal)
  • ProfileQueryBuilder unit tests cover first-N sample and count SQL generation
  • ProfilingManager unit tests cover schema retrieval and error handling with mocked workspace client

Blocked by

None — can start immediately.

User stories addressed

  • User story 1
  • User story 9
  • User story 12
  • User story 24
  • User story 25

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with PRD #172, especially its API Design, Error Handling, and Testing Decisions sections, then map the backend work across profiling.py, ProfileQueryBuilder, ProfilingManager, and profiling_routes.py. Review the TableProfiler entry point and the named unit and TestClient integration tests; done means the listed schema, count, sample, guardrail, error-state, and frontend acceptance criteria are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python, react, sql
Domain
api, databases, full-stack
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.