Bug: Layers fail to deploy correctly
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 1.2k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 52
Description
### Description:
When defining a layer in the template and using a makefile, the source dir is copied instead of the artifacts dir.
This changed recently, I think with the 1.110 update.
I am seeing this using the lambda python3.9 runtime.
### Steps to reproduce:
Define layer in template
```
Resources:
mnsLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: MyNotesLayer
Description: Common modules used by functions.
ContentUri: mnsLayer
CompatibleRuntimes:
- python3.9
RetentionPolicy: Delete
Metadata:
BuildMethod: makefile
```
Makefile-
```
PYVERS = 3.9
PROJDIR = $(HOME)/work/mytheta.dev/note_site/api
MNSLAYERDIR = $(PROJDIR)/mnsLayer
build-mnsLayer:
mkdir -p "$(ARTIFACTS_DIR)/python"
cd "$(MNSLAYERDIR)"
pip-${PYVERS} install --quiet --target="$(ARTIFACTS_DIR)/python" --upgrade -r requirements.txt
cp $(MNSLAYERDIR)/utils.py $(ARTIFACTS_DIR)/python/utils.py
cd $(ARTIFACTS_DIR)/python && python${PYVERS} -m compileall -o 2 -q .
```
### Observed result:
The source dir (mnsLayer) contains `Makefile`, `requirements.txt` and `utils.py` - these three files are found in the lambda runtime as `/opt/Makefile`, `/opt/requirements.txt` and `/opt/utils.py`
I can adjust my source to use the utils module with
```
import sys
sys.path.insert(0, '/opt')
import utils
```
but the extra modules in requirements are not found in the lambda runtime.
Locally, `.aws-sam/build/mnsLayer/python` contains the expected files, but they are not found in the lambda runtime.
### Expected result:
Previously this setup was installing `utils.py` so that it could be imported with `import utils` as well as extra modules in requirements being available.
### Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
1. OS: FreeBSD 13.2
2. `sam --version`: 1.110.0
3. AWS region: us-east-1
```
{
"version": "1.110.0",
"system": {
"python": "3.9.16",
"os": "FreeBSD-13.2-STABLE-amd64-64bit-ELF"
},
"additional_dependencies": {
"docker_engine": "Not available",
"aws_cdk": "2.123.0 (build a594112)",
"terraform": "Not available"
},
"available_beta_feature_env_vars": [
"SAM_CLI_BETA_FEATURES",
"SAM_CLI_BETA_BUILD_PERFORMANCE",
"SAM_CLI_BETA_TERRAFORM_SUPPORT",
"SAM_CLI_BETA_RUST_CARGO_LAMBDA"
]
}
```
`Add --debug flag to command you are running`
the portion of `sam build --debug` output that contains the layer build
```
2024-03-06 20:27:44,476 | Same Layer build definition found, adding layer (Previous: LayerBuildDefinition(mnsLayer,
/home/shane/work/mytheta.dev/note_site/api/mnsLayer, , 7781afc6-72c3-4a6f-a3e9-5d0cacb5b85a, makefile, ['python3.9'],
x86_64, {}), Current: LayerBuildDefinition(mnsLayer, /home/shane/work/mytheta.dev/note_site/api/mnsLayer, ,
748c9402-9a93-4d64-ad00-ad858ec8af9f, makefile, ['python3.9'], x86_64, {}), Layer:
)
2024-03-06 20:27:44,483 | Building layer 'mnsLayer'
2024-03-06 20:27:44,485 | Loading workflow module 'aws_lambda_builders.workflows'
2024-03-06 20:27:44,489 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided',
dependency_manager=None, application_framework=None)'
2024-03-06 20:27:44,493 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet',
dependency_manager='cli-package', application_framework=None)'
2024-03-06 20:27:44,496 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go',
dependency_manager='modules', application_framework=None)'
2024-03-06 20:27:44,501 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java',
dependency_manager='gradle', application_framework=None)'
2024-03-06 20:27:44,505 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java',
dependency_manager='maven', application_framework=None)'
2024-03-06 20:27:44,508 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs',
dependency_manager='npm', application_framework=None)'
2024-03-06 20:27:44,512 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs',
dependency_manager='npm-esbuild', application_framework=None)'
2024-03-06 20:27:44,517 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python',
dependency_manager='pip', application_framework=None)'
2024-03-06 20:27:44,520 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby',
dependency_manager='bundler', application_framework=None)'
2024-03-06 20:27:44,524 | Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust',
dependency_manager='cargo', application_framework=None)'
2024-03-06 20:27:44,526 | Found workflow 'CustomMakeBuilder' to support capabilities 'Capability(language='provided',
dependency_manager=None, application_framework=None)'
2024-03-06 20:27:44,528 | Running workflow 'CustomMakeBuilder'
2024-03-06 20:27:44,530 | mnsLayer: Running CustomMakeBuilder:CopySource
2024-03-06 20:27:44,532 | Copying source file (/home/shane/work/mytheta.dev/note_site/api/mnsLayer/Makefile) to
destination (/tmp/tmpgeyred0a/Makefile)
2024-03-06 20:27:44,534 | Creating target folders at /tmp/tmpgeyred0a/__pycache__
2024-03-06 20:27:44,536 | Copying directory metadata from source
(/home/shane/work/mytheta.dev/note_site/api/mnsLayer/__pycache__) to destination (/tmp/tmpgeyred0a/__pycache__)
2024-03-06 20:27:44,538 | Copying source file
(/home/shane/work/mytheta.dev/note_site/api/mnsLayer/__pycache__/utils.cpython-39.pyc) to destination
(/tmp/tmpgeyred0a/__pycache__/utils.cpython-39.pyc)
2024-03-06 20:27:44,540 | Copying source file (/home/shane/work/mytheta.dev/note_site/api/mnsLayer/utils.py) to
destination (/tmp/tmpgeyred0a/utils.py)
2024-03-06 20:27:44,542 | Copying source file (/home/shane/work/mytheta.dev/note_site/api/mnsLayer/requirements.txt) to
destination (/tmp/tmpgeyred0a/requirements.txt)
2024-03-06 20:27:44,544 | CustomMakeBuilder:CopySource succeeded
2024-03-06 20:27:44,545 | mnsLayer: Running CustomMakeBuilder:MakeBuild
2024-03-06 20:27:44,547 | mnsLayer: Current Artifacts Directory :
/home/shane/work/mytheta.dev/note_site/api/.aws-sam/build/mnsLayer
2024-03-06 20:27:44,548 | executing Make: ['make', '--makefile',
'/home/shane/work/mytheta.dev/note_site/api/mnsLayer/Makefile', 'build-mnsLayer']
`/home/shane/work/mytheta.dev/note_site/api/mnsLayer/Makefile' is up to date.
mkdir -p "/home/shane/work/mytheta.dev/note_site/api/.aws-sam/build/mnsLayer/python"
cd "/home/shane/work/mytheta.dev/note_site/api/mnsLayer"
pip-3.9 install --quiet --target="/home/shane/work/mytheta.dev/note_site/api/.aws-sam/build/mnsLayer/python" --upgrade -r requirements.txt
cp /home/shane/work/mytheta.dev/note_site/api/mnsLayer/utils.py /home/shane/work/mytheta.dev/note_site/api/.aws-sam/build/mnsLayer/python/utils.py
cd /home/shane/work/mytheta.dev/note_site/api/.aws-sam/build/mnsLayer/python && python3.9 -m compileall -o 2 -q .
2024-03-06 20:27:59,245 | CustomMakeBuilder:MakeBuild succeeded
```
Contributor guide
Assessment
This issue has not been assessed yet.