dbt-labs / dbt-labs/dbt-adapters
[CT-2027] [Feature] Standardize `setup.py` across adapter repos
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this your first time submitting a feature request?
- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
### Describe the feature
Loving the changes that @mikealfare made to [setup.py in Redshift](https://github.com/dbt-labs/dbt-redshift/blob/471bb1404536f8f55b18afba30db84443c17dbcb/setup.py).
It seems like we should standardize and make similar changes in the other adapters!
- [dbt-bigquery/setup.py](https://github.com/dbt-labs/dbt-bigquery/blob/edca37abfa21d492b75f1fa03d20e51da55304bb/setup.py)
- [dbt-snowflake/setup.py](https://github.com/dbt-labs/dbt-snowflake/blob/471d07e8b09bfdbee0ec35a7d69bf85b8dc946ca/setup.py)
- [dbt-spark/setup.py](https://github.com/dbt-labs/dbt-spark/blob/9241c385b1561e58aa262eff7c4782d2a174faf1/setup.py)
I'm not sure if they would apply to Postgres or not 🤷 -- someone would need to assess:
- [postgres/setup.py](https://github.com/dbt-labs/dbt-core/blob/1a6e4a00c7be2ea88492192de51d190a46c19561/plugins/postgres/setup.py)
Bonus suggestion:
- put the definition of `setup.py` [here](https://github.com/dbt-labs/dbt-database-adapter-scaffold/blob/1b7359ad805de276faabfeca0e83d83e8718715c/%7B%7Bcookiecutter.project_name%7D%7D/setup.py) and then just use that to stamp out instances for each adapter
### Describe alternatives you've considered
We _could_ allow the structure and contents of setup.py to drift. But I don't think it's _necessary_ for them to drift. After a detailed assessment, nearly everything I saw was **undifferentiated** across adapters.
### Who will this benefit?
One benefit:
- My assumption is that some maintainers of the adapters listed [here](https://docs.getdbt.com/docs/supported-data-platforms) largely take their `setup.py` from an adapter maintained by dbt Labs, and it would be nice for them to get the "latest and greatest" version.
Another benefit:
- Folks have run into problems that standardized setup.py could have prevented:
- Example: https://github.com/starburstdata/dbt-trino/issues/221
### Are you interested in contributing this feature?
Happy to help however -- pretty easy to open PRs for these
### Anything else?
See below for a templated version of `setup.py` (that may itself be stale already :shrug:). It would be very easy to convert this for usage by [Copier](https://copier.readthedocs.io/en/stable/), [Cruft](https://cruft.github.io/cruft/), [Cookiecutter](https://cookiecutter.readthedocs.io/en/latest/), [Cookieninja](https://cookieninja.readthedocs.io/en/latest/), etc.
Cookiecutter template for
`setup.py`
```python
#!/usr/bin/env python
import sys
if sys.version_info < (3, 7):
print("Error: dbt does not support this version of Python.")
print("Please upgrade to Python 3.7 or higher.")
sys.exit(1)
try:
from setuptools import find_namespace_packages
except ImportError:
print("Error: dbt requires setuptools v40.1.0 or higher.")
print('Please upgrade setuptools with "pip install --upgrade setuptools" and try again')
sys.exit(1)
from pathlib import Path
from setuptools import setup
# pull the long description from the README
README = Path(__file__).parent / "README.md"
# used for this adapter's version and in determining the compatible dbt-core version
VERSION = Path(__file__).parent / "dbt/adapters/{{ cookiecutter.directory_name }}/__version__.py"
def _plugin_version() -> str:
"""
Pull the package version from the main package version file
"""
attributes = {}
exec(VERSION.read_text(), attributes)
return attributes["version"]
def _core_patch(plugin_patch: str):
"""
Determines the compatible dbt-core patch given this plugin's patch
Args:
plugin_patch: the version patch of this plugin
"""
pre_release_phase = "".join([i for i in plugin_patch if not i.isdigit()])
if pre_release_phase:
if pre_release_phase not in ["a", "b", "rc"]:
raise ValueError(f"Invalid prerelease patch: {plugin_patch}")
return f"0{pre_release_phase}1"
return "0"
# require a compatible minor version (~=) and prerelease if this is a prerelease
def _core_version(plugin_version: str = _plugin_version()) -> str:
"""
Determine the compatible dbt-core version given this plugin's version.
We assume that the plugin must agree with `dbt-core` down to the minor version.
Args:
plugin_version: the version of this plugin, this is an argument in case we ever want to unit test this
"""
try:
major, minor, plugin_patch = plugin_version.split(".")
except ValueError:
raise ValueError(f"Invalid version: {plugin_version}")
return f"{major}.{minor}.{_core_patch(plugin_patch)}"
setup(
name="{{ cookiecutter.project_name }}",
version=_plugin_version(),
description="The {{ cookiecutter.platform_short_name }} adapter plugin for dbt",
long_description=README.read_text(),
long_description_content_type="text/markdown",
author="dbt Labs",
author_email="info@dbtlabs.com",
url="[https://github.com/dbt-labs/{{](https://github.com/dbt-labs/%7B%7B) cookiecutter.project_name }}",
packages=find_namespace_packages(include=["dbt", "dbt.*"]),
include_package_data=True,
install_requires=[
f"dbt-core~={_core_version()}",
],
zip_safe=False,
classifiers=[
"Development Status :: 5 - Production/Stable",
"License :: OSI Approved :: Apache Software License",
"Operating System :: Microsoft :: Windows",
"Operating System :: MacOS :: MacOS X",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
],
python_requires=">=3.7",
)
```
Contributor guide
Assessment
This issue has not been assessed yet.