elastic / elastic/support-diagnostics
Docker emulation as podman
- Dominant language
- Java
- Stars
- 304
- Forks
- 175
- PR merge metrics
- No merged PRs in 30d
Description
Hi there --
I'm currently going back and forth with support on some issues requiring diagnostic bundles. I was able to generate a bundle, but then I wasn't able to scrub it (full output below).
We are on a RHEL8 based system, and I did some testing and it turns out that the diagnostics.sh script is creating directories that include `Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.` in the directory name below. This is because podman is the container tool of choice on RHEL 8 and docker emits that message when running.
If I unzip, manually remove those files, and re-zip then the scrub completed successfully.
```
[root@foo:diagnostics-8.5.0]# ./diagnostics.sh --type logstash-local
Using /bin/java as Java Runtime
Using -Xms256m -Xmx2000m for options.
ERROR StatusConsoleListener Unable to locate appender "diag" for logger config "root"
Processing diagnosticInputs...
Creating temp directory: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics
Configuring log file.
Diagnostic logger reconfigured for inclusion into archive
Checking for diagnostic version updates.
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_version.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_node.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_node_stats.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_nodes_hot_threads.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_plugins.json
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_diagnostic_flow_metrics.html
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/top.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/netstat.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/ss.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/process-list.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/top_threads.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/uname.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/cpu.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/iostat.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/sar.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/sysctl.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/dmesg.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/dmesg_t.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/huge_pages.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/cpu_governor.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/limits.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/proc-limit.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/readahead.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-info.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-ps-all.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg..txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg..txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg..txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-c169fbf99e50.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-c169fbf99e50.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-c169fbf99e50.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-6a42a225628a.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-6a42a225628a.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-6a42a225628a.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-4cc674cb49a8.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-4cc674cb49a8.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-4cc674cb49a8.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-a1a3b2c64001.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-a1a3b2c64001.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-a1a3b2c64001.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-2ccc615e3f66.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-2ccc615e3f66.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-2ccc615e3f66.txt
Writing diagnostic manifest.
Identified Docker installations - bypassed log collection and some system calls.
Closing loggers.
Archiving diagnostic results.
Archive: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics-20231116-075654.zip was created
Deleted directory: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics.
[root@foo:diagnostics-8.5.0]# ./scrub.sh -i logstash-local-diagnostics-20231116-075654.zip
Using /bin/java as Java Runtime
Using -Xms8g -Xmx8g for options.
ERROR StatusConsoleListener Unable to locate appender "diag" for logger config "root"
Processing diagnosticInputs...
Using version: 8.5.0 of diagnostic-utiliy
Deleted directory: /root/lee1001/diagnostics-8.5.0/scrubbed-logstash-local-diagnostics-20231116-075654.
Diagnostic logger reconfigured for inclusion into archive
Threadpool configured with 64 workers.
Closing loggers.
Deleted directory: /root/lee1001/diagnostics-8.5.0/scrubbed-logstash-local-diagnostics-20231116-075654.
Fatal error occurred: Could not scrub archive. Check diagnostics.log in the archive file for more detail.
```
When I tried creating that `/etc/containers/nodocker` file to suppress the message, and re-run the process, it didn't create directories with those names, but it still errored out on the scrubbing:
```
[root@foo:diagnostics-8.5.0]# ./diagnostics.sh --type logstash-local
Using /bin/java as Java Runtime
Using -Xms256m -Xmx2000m for options.
ERROR StatusConsoleListener Unable to locate appender "diag" for logger config "root"
Processing diagnosticInputs...
Creating temp directory: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics
Configuring log file.
Diagnostic logger reconfigured for inclusion into archive
Checking for diagnostic version updates.
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_version.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_node.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_node_stats.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_nodes_hot_threads.json
Results written to: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_plugins.json
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/logstash_diagnostic_flow_metrics.html
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/top.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/netstat.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/ss.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/process-list.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/top_threads.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/uname.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/cpu.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/iostat.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/sar.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/sysctl.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/dmesg.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/dmesg_t.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/huge_pages.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/cpu_governor.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/limits.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/proc-limit.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/syscalls/readahead.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-info.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-ps-all.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-c169fbf99e50.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-c169fbf99e50.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-c169fbf99e50.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-6a42a225628a.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-6a42a225628a.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-6a42a225628a.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-4cc674cb49a8.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-4cc674cb49a8.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-4cc674cb49a8.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-a1a3b2c64001.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-a1a3b2c64001.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-a1a3b2c64001.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-logs-2ccc615e3f66.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-top-2ccc615e3f66.txt
Writing to /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics/docker/docker-inspect-2ccc615e3f66.txt
Writing diagnostic manifest.
Identified Docker installations - bypassed log collection and some system calls.
Closing loggers.
Archiving diagnostic results.
Archive: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics-20231116-081429.zip was created
Deleted directory: /root/lee1001/diagnostics-8.5.0/logstash-local-diagnostics.
[root@foo:diagnostics-8.5.0]# ./scrub.sh -i logstash-local-diagnostics-20231116-081429.zip
Using /bin/java as Java Runtime
Using -Xms8g -Xmx8g for options.
ERROR StatusConsoleListener Unable to locate appender "diag" for logger config "root"
Processing diagnosticInputs...
Using version: 8.5.0 of diagnostic-utiliy
Deleted directory: /root/lee1001/diagnostics-8.5.0/scrubbed-logstash-local-diagnostics-20231116-081429.
Diagnostic logger reconfigured for inclusion into archive
Threadpool configured with 64 workers.
Closing loggers.
Deleted directory: /root/lee1001/diagnostics-8.5.0/scrubbed-logstash-local-diagnostics-20231116-081429.
Fatal error occurred: Could not scrub archive. Check diagnostics.log in the archive file for more detail.
```
Any thoughts on what is going on here / how I might be able to solve this?
Contributor guide
Assessment
This issue has not been assessed yet.