dssg / dssg/triage

Look at updating SQLAlchemy to 1.4.x

Open
#894 1 comment 0 reactions 0 assignees View on GitHub
dependencies
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

This may involve some considerable refactoring, especially of unit tests, but by way of documentation, here are a couple of notes to consider from discussion with @jtwalsh0 and @ecsalomon on slack:
- 1.4 strongly encourages/demands transaction blocks. If you use ohio's pg_copy_to, it will create the table but not insert any data unless you add an extra context manager. e.g.
```
with engine.connect() as conn:
with conn.begin():
df.pg_copy_to(...)
```
- `connect().execute('sql here')` either requires a `text()` wrapper around the sql or switch to `exec_driver_sql` to bind to parameters
- The major changes we need to make for 1.4 are rewriting the test suite, which has several patterns for mocking the database contents, and sqlalchemy 1.4 _did not like_ most of them, and the errors were state based -- which tests fail depend on which tests ran before 😬

Contributor guide

Open the contributing guide

Research direction

Start by inventorying SQLAlchemy usage and the unit-test suite, especially pg_copy_to calls, transaction handling, raw execute calls, and database mocks. Run the tests to identify state-dependent failures. Done means the project works with SQLAlchemy 1.4.x, transaction and SQL execution patterns are compatible, and the test suite passes consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlalchemy
Domain
backend, databases
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.