dlt-hub / dlt-hub/dlt

Running dlt in Databricks

Open
#3,569 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
5.9k
Forks
600
Avg merge
1d 14h
Merged PRs (30d)
38

Description

### Documentation description

I am trying to use dlt within a Databricks environment.
However, the dlt import fails due to a name clash with the Databricks' delta live tables.
Issue detailed here https://dlthub.notion.site/Databricks-notebook-instructions-980832a90fab4a98b6c8aa010d47646e.

Also tried the init script tweak from this reddit thread relating to the above guide. Didn't work.
https://www.reddit.com/r/dataengineering/comments/1g2wl3k/using_data_load_tool_dlthub_together_with/

However, the workaround here does not work.

There is no surfaced error message or log, just that the init script fails.

I would like the guide linked above to be updated or replaced.

### Are you a dlt user?

Yes, I'm already a dlt user.

Contributor guide

Open the contributing guide

Research direction

Start with the linked Databricks notebook instructions and the referenced init-script workaround, then reproduce the dlt import name clash and the init-script failure while collecting available logs. Done means the guide is updated or replaced with a verified Databricks procedure and explains how to resolve the conflict.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.