airbytehq / airbytehq/PyAirbyte

Bug: PermissionError on config migration write - temp files created read-only (0o444)

Open
#1,012 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
344
Forks
77
Avg merge
1d 11h
Merged PRs (30d)
35

Description

## Description

When running `source-facebook-marketing` (v5.0.0) via PyAirbyte in a Docker-in-Docker setup, the connector's `MigrateIncludeDeletedToStatusFilters.migrate()` fails with a `PermissionError` when attempting to write back the migrated config to the temp file.

Related to #656 (same root cause, different connector).

> **Note:** This issue does **not** reproduce on macOS with Docker Desktop, because Docker Desktop's VirtioFS/grpcfuse layer relaxes Unix permission enforcement on bind-mounted volumes. It manifests on **Linux** environments (e.g. ECS, CI) where permissions are strictly enforced.

## Error

```
PermissionError: [Errno 13] Permission denied: '/airbyte/tmp/tmphpz2m88c.json'
File "/usr/local/lib/python3.11/site-packages/airbyte_cdk/connector.py", line 67, in write_config
with open(config_path, "w") as fh:
```

## Root cause

In `airbyte/_util/temp_files.py`, temp config files are created with **read-only** permissions for all users (`0o444`):

```python
# Grant "read" permission to all users
Path(temp_file.name).chmod(stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH)
```

In a Docker-in-Docker setup on Linux:
- The outer container (PyAirbyte) runs as `root` and creates the temp file with `0o444`
- The inner connector container runs as `airbyte` user (uid 1000)
- The connector's config migration (`MigrateIncludeDeletedToStatusFilters`) tries to **write** to the temp file → `PermissionError`

PR #637 introduced world-readable permissions on temp files but did not include write permissions, so config migrations that need to write back to the temp file still fail on Linux.

## Possible fix (proposal)

One option could be to also grant write permissions to all users on the temp file:

```python
Path(temp_file.name).chmod(
stat.S_IRUSR | stat.S_IWUSR |
stat.S_IRGRP | stat.S_IWGRP |
stat.S_IROTH | stat.S_IWOTH
) # 0o666
```

There may be other approaches (e.g. running the inner container as the same user, or a different temp file strategy) — happy to discuss what fits best.

## Environment

- `airbyte`: 0.44.1
- Connector: `airbyte/source-facebook-marketing:5.0.0`
- Execution context: Docker-in-Docker on Linux (ECS)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.