cloudtools / cloudtools/stacker

file lookup should not be nested

Open
#510 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
707
Forks
162
PR merge metrics
No merged PRs in 30d

Description

Currently all lookups are nested, so after stacker has resolved some vars, it will load the resolved var and try to do another round of lookup until it cannot find any lookups in the produced value. However, this has brought some issues in my user case. I'm using file lookup to reference some file content for userdata, which in turn contains some bash logic. stacker is expected to resolve the first lookup, but is should not look inside my userdata script and try to replace the bash vars inside of it(which will fail and raise an exception).

I think nested lookup is a great feature, it is really powerful. However, for the case of file lookup, I think it should not try to resolve the loaded content. Or at least, stacker should provide an option not to go nested if the user want to.

I cannot find a way to implement this without a lot of change in the codebase. The lookup is nested because of the infinite loop in `variables.py`, the `resolve_lookups` function in `registry.py` has no way to reset `self.lookups` so the loop could be broken. Perhaps the lookup handlers could raise a special exception and we can handle the exception in `variables.py` and break there, but I think that's for the devs to decide.

My local setup:

config file:
```
CloudFormationInitConfigSetsFile: ${file plain:file://../files/userdata.json}
```

userdata.json
```
...
"/usr/local/bin/foobar export --namespace ${NAMESPACE} > /opt/foo/foo.conf\n\n",
...
```

It would produce the following error message:

```
stacker.exceptions.FailedVariableLookup: Couldn't resolve lookups in variable `CloudFormationInitConfigSetsFile`. Unknown lookup type: "None"
```

It happened because stacker had created a new lookup in runtime and it is:

```
set([Lookup(type=None, input='NAMESPACE', raw='NAMESPACE')])
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.