docker swarm - deploying non external secrets without writing them to disk

Description

I am deploying a stack from an azure devops pipeline.
When the pipeline runs, azure devops exposes "secrets" e.g from KeyVault, as variables and environment variables to tasks that run during the CI/CD pipeline on the build agent.

Right now, I have to either:-

• Write these sensitive values to disk on the build agent prior to doing docker stack deploy so that my stack (compose file) can own these secrets - e.g not external. This is less secure asthe next approach as I am forced to write them out to disk.
• Deploy them as external secrets passing them as std in to docker stack secret create. Then in my compose file, have the secrets as external

The choice over whether to use external secrets or not should not really be about having a secure method of deployment for said secret. It should be about whether it makes sense to manage the secret externally to the stack, or whether the stack owns the secret and the secret should be removed when the stack is removed. For secrets that are not generally used outside the stack it wouldn't make sense for me to want to manage them as exetrnal secrets. Irrespective of which model makes sense (external vs non external), I should be able to deploy that secret without writing it to disk. This is not the case however, as I can only deploy a secret held in memory as an external secret via docker secret create using stdin but I can't do that for the secrets that are non external. So this asymmetry in secret deployment capability is pushing me to use one model over the other even though that should not be the main driver.

There has been some related work:-

• to add raw here: I would not be keen on using raw if it meant I still have to write the secret value to disk in the compose file first as I may as well just write the values to secret files first and reference them as files in compose. However - if raw supports environment variable interpolation then this would solve the issue for me as it would be a roundabout way of being able to deploy non external secrets by supplying an in memory value at deployment time which is a step up in security from today.
• It looks like the compose spec has added an environment property for secrets - which looks promising but docker stack deploy does not yet support the compose spec

Start with the docker stack deploy secret-handling entry point and compare it with the Compose environment secret support described in the issue. Review issue #2527 and PR #712, then define done as deploying non-external secrets from pipeline-held values without writing them to disk while preserving stack-owned secret lifecycle.