Feature Request: Support Custom K3s/containerd Configuration, Proxy/Mirror Settings, Volume Mounts, and Environment Variables for Gateway Container

Problem Statement

🐛 Bug Description / Feature Request

Current Pain Points

When starting the OpenShell gateway container, the following critical limitations severely hinder deployment in Chinese domestic network environments and enterprise private cloud scenarios:

1. No support for custom K3s/containerd startup parameters

   • Cannot pass --pause-image, --kubelet-arg, --containerd-config or other K3s startup arguments
   • Cannot customize containerd configuration (e.g., registry mirrors, sandbox image)
   • K3s/containerd in the container completely ignores host-side proxy, mirror, and pre-downloaded image configurations

2. No support for custom proxy configuration inside the container

   • Host-side system proxy/Docker proxy is not inherited by the container's K3s/containerd process
   • Cannot pass HTTP_PROXY/HTTPS_PROXY environment variables to the container or K3s service
   • All image pull requests (e.g., rancher/mirrored-pause:3.6, K3s core components) force access to public Docker Hub/GHCR, resulting in network timeouts and startup failures in domestic networks

3. No support for custom image registry/mirror configuration

   • Cannot configure domestic mainstream mirror sites (e.g., Alibaba Cloud, DaoCloud, Nanjing University Mirror Station) for containerd
   • Cannot configure private enterprise registry authentication
   • Pre-downloaded images on the host cannot be recognized and used by K3s in the container (requires manual ctr -n k8s.io import after container startup, which is not sustainable)

4. No support for custom volume mounts

   • Cannot mount host directories (e.g., /etc/rancher/k3s, /var/lib/rancher/k3s/agent/images) into the container
   • Cannot persist K3s configuration, image cache, or custom manifests
   • Container is ephemeral: startup failure triggers automatic container deletion, making troubleshooting impossible

5. No support for passing environment variables to the container

   • Cannot inject HTTP_PROXY, NO_PROXY, REGISTRY_MIRROR or other environment variables
   • Cannot customize runtime behavior of K3s/containerd in the container

🎯 Expected Behavior

Add the following features to the openshell gateway start command to support flexible deployment in domestic/enterprise environments:

1. Support custom K3s startup arguments

   • Add a --k3s-arg flag to pass arbitrary K3s server/agent startup parameters (e.g., --k3s-arg "--pause-image=rancher/mirrored-pause:3.6", --k3s-arg "--kubelet-arg=fail-cgroupv1=false")
   • Support specifying server/agent scope (e.g., --k3s-arg "--pause-image=xxx@server:*")

2. Support custom containerd configuration

   • Allow mounting a custom config.toml for containerd to configure registry mirrors, sandbox image, and proxy settings
   • Add a --containerd-config flag to specify a host-side config file to be injected into the container

3. Support proxy configuration inheritance and injection

   • Inherit host-side Docker proxy settings by default
   • Add --http-proxy/--https-proxy/--no-proxy flags to explicitly set proxy for the container and K3s/containerd processes
   • Inject proxy environment variables into the container's system environment and K3s service

4. Support custom volume mounts

   • Add a --volume flag (compatible with Docker's -v syntax) to mount host directories into the container
   • Example: --volume /etc/rancher/k3s:/etc/rancher/k3s:ro, --volume /opt/k3s-images:/var/lib/rancher/k3s/agent/images:ro
   • Critical: Prevent automatic container deletion on startup failure, or add a --keep-failed-container flag to retain the container for troubleshooting

5. Support custom image registry/mirror configuration

   • Add a --registry-mirror flag to configure domestic mirror sites for containerd
   • Add --registry-username/--registry-password flags to support private enterprise registry authentication
   • Automatically load pre-downloaded images from /var/lib/rancher/k3s/agent/images in the container (already a K3s feature, but ensure it works in OpenShell)

6. Support passing arbitrary environment variables

   • Add an --env flag to pass custom environment variables to the container (e.g., --env HTTP_PROXY=http://proxy:7897, --env NO_PROXY=localhost,127.0.0.1,10.0.0.0/8)

📊 Use Case Scenario

• Chinese domestic network: Users need to configure domestic mirror sites and proxies to avoid Docker Hub/GHCR timeouts, which is the root cause of the common K8s namespace not ready startup failure (as shown in #722, #716)
• Enterprise private cloud: Users need to use private registries, custom K3s configurations, and persistent volume mounts for production deployment
• Offline deployment: Users need to pre-download all images and mount them into the container for fully offline startup

🧪 Current Workaround (Fragile and Unsustainable)

Currently, users have to use hacky workarounds that are not production-ready:

1. Manually monitor container startup, then manually docker cp images and ctr -n k8s.io import them
2. Manually inject proxy environment variables into the container after startup
3. Manually modify containerd config inside the container before K3s starts
4. These steps are error-prone, cannot be automated, and fail if the container is deleted on startup failure

💡 Suggested Implementation

• Extend the openshell gateway start CLI with the flags mentioned above
• Ensure all configurations are properly injected into the container before K3s/containerd starts
• Maintain backward compatibility: all new flags are optional, default behavior remains unchanged
• Align with K3s native configuration practices to avoid breaking changes

📎 Related Issues

• #722: Error: × K8s namespace not ready — timed out waiting for namespace 'openshell' to exist
• #716: OpenShell gateway start fails on Ubuntu 22.04.5 ARM64: K8s namespace not ready, flannel subnet.env missing

---

📌 Additional Notes

This feature is critical for adoption in Chinese domestic and enterprise environments, as the current "black box" container design makes it impossible to deploy OpenShell reliably without public network access.

Proposed Design

🐛 Bug Description / Feature Request

Current Pain Points

When starting the OpenShell gateway container, the following critical limitations severely hinder deployment in Chinese domestic network environments and enterprise private cloud scenarios:

1. No support for custom K3s/containerd startup parameters

   • Cannot pass --pause-image, --kubelet-arg, --containerd-config or other K3s startup arguments
   • Cannot customize containerd configuration (e.g., registry mirrors, sandbox image)
   • K3s/containerd in the container completely ignores host-side proxy, mirror, and pre-downloaded image configurations

2. No support for custom proxy configuration inside the container

   • Host-side system proxy/Docker proxy is not inherited by the container's K3s/containerd process
   • Cannot pass HTTP_PROXY/HTTPS_PROXY environment variables to the container or K3s service
   • All image pull requests (e.g., rancher/mirrored-pause:3.6, K3s core components) force access to public Docker Hub/GHCR, resulting in network timeouts and startup failures in domestic networks

3. No support for custom image registry/mirror configuration

   • Cannot configure domestic mainstream mirror sites (e.g., Alibaba Cloud, DaoCloud, Nanjing University Mirror Station) for containerd
   • Cannot configure private enterprise registry authentication
   • Pre-downloaded images on the host cannot be recognized and used by K3s in the container (requires manual ctr -n k8s.io import after container startup, which is not sustainable)

4. No support for custom volume mounts

   • Cannot mount host directories (e.g., /etc/rancher/k3s, /var/lib/rancher/k3s/agent/images) into the container
   • Cannot persist K3s configuration, image cache, or custom manifests
   • Container is ephemeral: startup failure triggers automatic container deletion, making troubleshooting impossible

5. No support for passing environment variables to the container

   • Cannot inject HTTP_PROXY, NO_PROXY, REGISTRY_MIRROR or other environment variables
   • Cannot customize runtime behavior of K3s/containerd in the container

🎯 Expected Behavior

Add the following features to the openshell gateway start command to support flexible deployment in domestic/enterprise environments:

1. Support custom K3s startup arguments

   • Add a --k3s-arg flag to pass arbitrary K3s server/agent startup parameters (e.g., --k3s-arg "--pause-image=rancher/mirrored-pause:3.6", --k3s-arg "--kubelet-arg=fail-cgroupv1=false")
   • Support specifying server/agent scope (e.g., --k3s-arg "--pause-image=xxx@server:*")

2. Support custom containerd configuration

   • Allow mounting a custom config.toml for containerd to configure registry mirrors, sandbox image, and proxy settings
   • Add a --containerd-config flag to specify a host-side config file to be injected into the container

3. Support proxy configuration inheritance and injection

   • Inherit host-side Docker proxy settings by default
   • Add --http-proxy/--https-proxy/--no-proxy flags to explicitly set proxy for the container and K3s/containerd processes
   • Inject proxy environment variables into the container's system environment and K3s service

4. Support custom volume mounts

   • Add a --volume flag (compatible with Docker's -v syntax) to mount host directories into the container
   • Example: --volume /etc/rancher/k3s:/etc/rancher/k3s:ro, --volume /opt/k3s-images:/var/lib/rancher/k3s/agent/images:ro
   • Critical: Prevent automatic container deletion on startup failure, or add a --keep-failed-container flag to retain the container for troubleshooting

5. Support custom image registry/mirror configuration

   • Add a --registry-mirror flag to configure domestic mirror sites for containerd
   • Add --registry-username/--registry-password flags to support private enterprise registry authentication
   • Automatically load pre-downloaded images from /var/lib/rancher/k3s/agent/images in the container (already a K3s feature, but ensure it works in OpenShell)

6. Support passing arbitrary environment variables

   • Add an --env flag to pass custom environment variables to the container (e.g., --env HTTP_PROXY=http://proxy:7897, --env NO_PROXY=localhost,127.0.0.1,10.0.0.0/8)

📊 Use Case Scenario

• Chinese domestic network: Users need to configure domestic mirror sites and proxies to avoid Docker Hub/GHCR timeouts, which is the root cause of the common K8s namespace not ready startup failure (as shown in #722, #716)
• Enterprise private cloud: Users need to use private registries, custom K3s configurations, and persistent volume mounts for production deployment
• Offline deployment: Users need to pre-download all images and mount them into the container for fully offline startup

🧪 Current Workaround (Fragile and Unsustainable)

Currently, users have to use hacky workarounds that are not production-ready:

1. Manually monitor container startup, then manually docker cp images and ctr -n k8s.io import them
2. Manually inject proxy environment variables into the container after startup
3. Manually modify containerd config inside the container before K3s starts
4. These steps are error-prone, cannot be automated, and fail if the container is deleted on startup failure

💡 Suggested Implementation

• Extend the openshell gateway start CLI with the flags mentioned above
• Ensure all configurations are properly injected into the container before K3s/containerd starts
• Maintain backward compatibility: all new flags are optional, default behavior remains unchanged
• Align with K3s native configuration practices to avoid breaking changes

📎 Related Issues

• #722: Error: × K8s namespace not ready — timed out waiting for namespace 'openshell' to exist
• #716: OpenShell gateway start fails on Ubuntu 22.04.5 ARM64: K8s namespace not ready, flannel subnet.env missing

---

📌 Additional Notes

This feature is critical for adoption in Chinese domestic and enterprise environments, as the current "black box" container design makes it impossible to deploy OpenShell reliably without public network access.

Alternatives Considered

🐛 Bug Description / Feature Request

Current Pain Points

When starting the OpenShell gateway container, the following critical limitations severely hinder deployment in Chinese domestic network environments and enterprise private cloud scenarios:

1. No support for custom K3s/containerd startup parameters

   • Cannot pass --pause-image, --kubelet-arg, --containerd-config or other K3s startup arguments
   • Cannot customize containerd configuration (e.g., registry mirrors, sandbox image)
   • K3s/containerd in the container completely ignores host-side proxy, mirror, and pre-downloaded image configurations

2. No support for custom proxy configuration inside the container

   • Host-side system proxy/Docker proxy is not inherited by the container's K3s/containerd process
   • Cannot pass HTTP_PROXY/HTTPS_PROXY environment variables to the container or K3s service
   • All image pull requests (e.g., rancher/mirrored-pause:3.6, K3s core components) force access to public Docker Hub/GHCR, resulting in network timeouts and startup failures in domestic networks

3. No support for custom image registry/mirror configuration

   • Cannot configure domestic mainstream mirror sites (e.g., Alibaba Cloud, DaoCloud, Nanjing University Mirror Station) for containerd
   • Cannot configure private enterprise registry authentication
   • Pre-downloaded images on the host cannot be recognized and used by K3s in the container (requires manual ctr -n k8s.io import after container startup, which is not sustainable)

4. No support for custom volume mounts

   • Cannot mount host directories (e.g., /etc/rancher/k3s, /var/lib/rancher/k3s/agent/images) into the container
   • Cannot persist K3s configuration, image cache, or custom manifests
   • Container is ephemeral: startup failure triggers automatic container deletion, making troubleshooting impossible

5. No support for passing environment variables to the container

   • Cannot inject HTTP_PROXY, NO_PROXY, REGISTRY_MIRROR or other environment variables
   • Cannot customize runtime behavior of K3s/containerd in the container

🎯 Expected Behavior

Add the following features to the openshell gateway start command to support flexible deployment in domestic/enterprise environments:

1. Support custom K3s startup arguments

   • Add a --k3s-arg flag to pass arbitrary K3s server/agent startup parameters (e.g., --k3s-arg "--pause-image=rancher/mirrored-pause:3.6", --k3s-arg "--kubelet-arg=fail-cgroupv1=false")
   • Support specifying server/agent scope (e.g., --k3s-arg "--pause-image=xxx@server:*")

2. Support custom containerd configuration

   • Allow mounting a custom config.toml for containerd to configure registry mirrors, sandbox image, and proxy settings
   • Add a --containerd-config flag to specify a host-side config file to be injected into the container

3. Support proxy configuration inheritance and injection

   • Inherit host-side Docker proxy settings by default
   • Add --http-proxy/--https-proxy/--no-proxy flags to explicitly set proxy for the container and K3s/containerd processes
   • Inject proxy environment variables into the container's system environment and K3s service

4. Support custom volume mounts

   • Add a --volume flag (compatible with Docker's -v syntax) to mount host directories into the container
   • Example: --volume /etc/rancher/k3s:/etc/rancher/k3s:ro, --volume /opt/k3s-images:/var/lib/rancher/k3s/agent/images:ro
   • Critical: Prevent automatic container deletion on startup failure, or add a --keep-failed-container flag to retain the container for troubleshooting

5. Support custom image registry/mirror configuration

   • Add a --registry-mirror flag to configure domestic mirror sites for containerd
   • Add --registry-username/--registry-password flags to support private enterprise registry authentication
   • Automatically load pre-downloaded images from /var/lib/rancher/k3s/agent/images in the container (already a K3s feature, but ensure it works in OpenShell)

6. Support passing arbitrary environment variables

   • Add an --env flag to pass custom environment variables to the container (e.g., --env HTTP_PROXY=http://proxy:7897, --env NO_PROXY=localhost,127.0.0.1,10.0.0.0/8)

📊 Use Case Scenario

• Chinese domestic network: Users need to configure domestic mirror sites and proxies to avoid Docker Hub/GHCR timeouts, which is the root cause of the common K8s namespace not ready startup failure (as shown in #722, #716)
• Enterprise private cloud: Users need to use private registries, custom K3s configurations, and persistent volume mounts for production deployment
• Offline deployment: Users need to pre-download all images and mount them into the container for fully offline startup

🧪 Current Workaround (Fragile and Unsustainable)

Currently, users have to use hacky workarounds that are not production-ready:

1. Manually monitor container startup, then manually docker cp images and ctr -n k8s.io import them
2. Manually inject proxy environment variables into the container after startup
3. Manually modify containerd config inside the container before K3s starts
4. These steps are error-prone, cannot be automated, and fail if the container is deleted on startup failure

💡 Suggested Implementation

• Extend the openshell gateway start CLI with the flags mentioned above
• Ensure all configurations are properly injected into the container before K3s/containerd starts
• Maintain backward compatibility: all new flags are optional, default behavior remains unchanged
• Align with K3s native configuration practices to avoid breaking changes

📎 Related Issues

• #722: Error: × K8s namespace not ready — timed out waiting for namespace 'openshell' to exist
• #716: OpenShell gateway start fails on Ubuntu 22.04.5 ARM64: K8s namespace not ready, flannel subnet.env missing

---

📌 Additional Notes

This feature is critical for adoption in Chinese domestic and enterprise environments, as the current "black box" container design makes it impossible to deploy OpenShell reliably without public network access.

Agent Investigation

No response

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request