Add Azure Key Vault support as a VaultDocProvider
- Dominant language
- Python
- Stars
- 5.9k
- Forks
- 600
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 38
Description
### Feature description
dlt currently supports retrieving configuration from secure vaults such as Google Secret Manager and Airflow Variables. The documentation also mentions that contributions for other vault integrations like Azure Key Vault are welcome.
i waned to propose adding Azure Key Vault support as a new VaultDocProvider.
Allow users to load dlt configuration and secrets from Azure Key Vault in the same way they can currently use Google Secret Manager.
This would provide a first-class secret management option for users deploying dlt on Azure.
### Are you a dlt user?
Yes, I'm already a dlt user.
### Use case
Organizations running workloads on Azure often store application secrets in Azure Key Vault and prefer not to duplicate credentials in environment variables or configuration files.
Currently, Azure users do not have a built-in vault integration equivalent to Google Secret Manager.
My case is simple, i'm using fabricspark as backbone of DBT and now for ingestion job i wanted to run DLT job inside fabric notebook and i'm already using AKS which will store source creds such as strip, jdbc, kafka, salesforce etc.
### Proposed solution
Implement an AzureKeyVaultDocProvider based on:
- [azure-keyvault-secrets](https://pypi.org/project/azure-keyvault-secrets/)
- [azure-identity](https://pypi.org/project/azure-identity/)
Authentication could use [DefaultAzureCredential](https://learn.microsoft.com/en-us/dotnet/api/azure.identity.defaultazurecredential?view=azure-dotnet), which supports:
- Local development via az login
- Service principals
- Managed identities
- CI/CD environments
This would follow Azure SDK best practices while keeping authentication configuration minimal.
Needs to check whether azure dependencies should be provided as optional extras.
### Related issues
I couldn't find an existing issue or active PR for Azure Key Vault support, but please let me know if I missed one.
Contributor guide
Research direction
Start by locating the existing Google Secret Manager VaultDocProvider and its tests, then compare how dlt configuration and secrets are loaded. Review the azure-keyvault-secrets and azure-identity integration and determine how optional dependencies are declared. Done means Azure Key Vault can provide configuration and secrets through the same provider mechanism, with authentication and dependency behavior covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python
- Domain
- cloud, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100