Keep container/-step alive across dags (when referencing as sub-dag)
- Dominant language
- Go
- Stars
- 4k
- Forks
- 332
- Avg merge
- 19h 26m
- Merged PRs (30d)
- 133
Description
Hello, I just started to play around with dagu, and I am really liking it so far, but I seemingly can't get the following scenario to work.
I would like to create a container or step that specifies spins up a postgres-db, and exposes the ports in the host network. Ideally, this container/step can be included as a sub-dag in other dags.
I've started from the postgres-example given in #1244, which works perfectly.
That example I've changed to the following dag named `postgres`:
```yaml
container:
image: postgres:17
env:
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=appdb
keepContainer: true
network: host
ports:
- 5432:5432
startup: entrypoint
waitFor: running
steps:
- name: check-ready
command: pg_isready -U postgres -d appdb
retryPolicy:
limit: 10
intervalSec: 1
```
From this [example](https://docs.dagu.io/writing-workflows/examples#keep-container-running) and the [note on the container-configuration](https://docs.dagu.io/reference/yaml#container-configuration) I assume that the container is kept alive atleast until the sub-dag is fully executed. The initial test with the example given in #1244 seems to confirm this.
However, if I call the postgres-dag in another dag - let's call it `postgres_create_table` - I seemingly can't connect to the postgres-db.
The dag-spec for 'postgres_create_table` is as follows:
```yaml
params:
- CREATE_TABLE: "'CREATE TABLE IF NOT EXISTS demo(id INT PRIMARY KEY, name TEXT);'"
env:
- PGPASSWORD: secret
steps:
- call: postgres.yaml
depends: []
- name: create-db
executor:
type: docker
config:
image: alpine
autoRemove: true
command: sh -c "apk add postgresql17-client --allow-untrusted && psql -U postgres -h localhost -d appdb -c ${CREATE_TABLE}"
```
The error message is as follows:
```
* Setting postgresql17 as the default version
psql: error: connection to server at "localhost" (::1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
```
If I run `netstat` instead of the `create-db`-step, it yields an empty list.
I also tried changing the `startup`-property of `posgtres`-dag to `keepalive` and manually trigger the entrypoint-script and the startup of the postgres-image. This fails either due to inappropriate rights (sudo-user starting postgres) or the `postgresql.conf` not being available after explicitly changing the user via `su -c postgres postgres` with the following error message:
```
postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory
```
Am I making an obvious mistake, or is it simply not possible as of now to keep a container/step alive across dags? Thanks for any response in advance!
Contributor guide
Assessment
This issue has not been assessed yet.