Using lamda to access RDS
- Dominant language
- Python
- Stars
- 11.1k
- Forks
- 1k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 2
Description
# Problem Statement
I am trying to use chalice to update a table in mysql based on files being uploaded to S3. For the moment, I am not even interested in the content of the files. I just want to insert of update a line in my db to say that a give file has been uploaded.
# Auto-generate roles
Chalice auto-generates a role with S3 and CloudWatch permission, but nothing about access to the DB.
# Attempted workarounds
There is configuration to use one's own role and permission. I noticed the following
* Chalice seems to expect a role named `my-project-dev`
* Even having created the a role named `my-project-dev`, but I get an error which say that the role can't be deleted.
```Creating deployment package.
Updating lambda function: top200-dev-update_metadata
Configuring S3 events in bucket top200-cleandata to function top200-dev-update_metadata
Deleting IAM role: top200-dev
Traceback (most recent call last):
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/deployer.py", line 342, in deploy
return self._deploy(config, chalice_stage_name)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/deployer.py", line 355, in _deploy
self._executor.execute(plan)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/executor.py", line 31, in execute
self._default_handler)(instruction)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/executor.py", line 43, in _do_apicall
result = method(**final_kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/awsclient.py", line 433, in delete_role
client.delete_role(RoleName=name)
File "/home/markchassy/.local/lib/python3.7/site-packages/botocore/client.py", line 276, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/botocore/client.py", line 586, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.DeleteConflictException: An error occurred (DeleteConflict) when calling the DeleteRole operation: Cannot delete entity, must detach all policies first.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/cli/__init__.py", line 512, in main
return cli(obj={})
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/cli/__init__.py", line 205, in deploy
deployed_values = d.deploy(config, chalice_stage_name=stage)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/deployer.py", line 344, in deploy
raise ChaliceDeploymentError(e)
chalice.deploy.deployer.ChaliceDeploymentError: ERROR - While deploying your chalice application, received the following error:
An error occurred (DeleteConflict) when calling the DeleteRole operation:
Cannot delete entity, must detach all policies first.
```
I tried to both specify the json for the permission `and` to have those permission already configured in AWS. Finally, it is unclear if permission should be in `policy.json` or `policy-dev.json`
Here is my config.json
```
{
"version": "2.0",
"app_name": "top200",
"stages": {
"dev": {
"api_gateway_stage": "top200",
"manage_iam_role": false,
"autogen_policy": false,
"iam_role_arn": "arn:aws:iam::439359573308:role/top200-dev"
}
}
}
```
Here is my policy.json
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:eu-west-1:439359573308:dbuser:arn:aws:rds:us-east-1:439359573308:db:top200-portal/top200admin"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"*"
],
"Sid": "5a1a5b736299483abb776aeee43f4a88"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}
```
Contributor guide
Research direction
Start with chalice/deploy/deployer.py and chalice/awsclient.py around role deletion, then inspect how manage_iam_role, autogen_policy, and iam_role_arn are handled for stages. Reproduce the deployment error and check the handling of policy.json versus policy-dev.json; done when deployment uses the intended role without the DeleteConflict error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, mysql, python
- Domain
- backend, cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100