elastic / elastic/elastic-package

docker-compose service deployer - Provide a volume to share data with elastic-agent

Open
#1,852 1 comment 2 reactions 0 assignees View on GitHub
stale
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

On Linux, bind mounting the host-based `SERVICE_LOGS_DIR` into a container can cause permissions problems for elastic-package when the service writes files as root. You can get failures like:

> Error: error running package system tests: could not complete test run: could not setup service: removing service logs failed: readdir failed (path: /opt/buildkite-agent/.elastic-package/tmp/service_logs): open /opt/buildkite-agent/.elastic-package/tmp/service_logs: permission denied

Instead of using a host bind-mount, elastic-package could create volume and mount it inside the elastic-agent container. Then when a service want to share logs with the elastic-agent, it can mount the volume instead of SERVICE_LOGS_DIR. This would look like

```yaml
# ./profiles/default/stack/docker-compose.yml
services:
elastic-agent:
volumes:
- elastic_agent_volume:/elastic_agent_volume

volumes:
elastic_agent_volume:
# Prevent docker-compose from prefixing the volume name.
name: elastic_agent_volume
```

Then in the integration deployment it refers to an external volume.

```yaml
# _dev/deploy/docker/docker-compose.yml
version: '3.0'
services:
gitlab:
image: gitlab/gitlab-ce:${SERVICE_VERSION}
volumes:
- elastic_agent_volume:/var/log/gitlab/gitlab-rails

volumes:
elastic_agent_volume:
# To indicate that docker-compose should not create this volume.
external: true
```

### Related

- #1341

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with profiles/default/stack/docker-compose.yml and _dev/deploy/docker/docker-compose.yml, then review related issue #1341. Reproduce the Linux permission failure described in the issue and compare the current SERVICE_LOGS_DIR sharing with the proposed named volume setup. Done means the elastic-agent and deployed service can share logs without the host bind-mount permission error.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, go
Domain
devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.