aws / aws/chalice

Using lamda to access RDS

Open
#1,345 17 comments 4 reactions 0 assignees View on GitHub
feature-request
Dominant language
Python
Stars
11.1k
Forks
1k
Avg merge
1d 22h
Merged PRs (30d)
2

Description

# Problem Statement
I am trying to use chalice to update a table in mysql based on files being uploaded to S3. For the moment, I am not even interested in the content of the files. I just want to insert of update a line in my db to say that a give file has been uploaded.

# Auto-generate roles
Chalice auto-generates a role with S3 and CloudWatch permission, but nothing about access to the DB.

# Attempted workarounds
There is configuration to use one's own role and permission. I noticed the following
* Chalice seems to expect a role named `my-project-dev`
* Even having created the a role named `my-project-dev`, but I get an error which say that the role can't be deleted.
```Creating deployment package.
Updating lambda function: top200-dev-update_metadata
Configuring S3 events in bucket top200-cleandata to function top200-dev-update_metadata
Deleting IAM role: top200-dev
Traceback (most recent call last):
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/deployer.py", line 342, in deploy
return self._deploy(config, chalice_stage_name)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/deployer.py", line 355, in _deploy
self._executor.execute(plan)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/executor.py", line 31, in execute
self._default_handler)(instruction)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/executor.py", line 43, in _do_apicall
result = method(**final_kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/awsclient.py", line 433, in delete_role
client.delete_role(RoleName=name)
File "/home/markchassy/.local/lib/python3.7/site-packages/botocore/client.py", line 276, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/botocore/client.py", line 586, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.DeleteConflictException: An error occurred (DeleteConflict) when calling the DeleteRole operation: Cannot delete entity, must detach all policies first.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/cli/__init__.py", line 512, in main
return cli(obj={})
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/cli/__init__.py", line 205, in deploy
deployed_values = d.deploy(config, chalice_stage_name=stage)
File "/home/markchassy/.local/lib/python3.7/site-packages/chalice/deploy/deployer.py", line 344, in deploy
raise ChaliceDeploymentError(e)
chalice.deploy.deployer.ChaliceDeploymentError: ERROR - While deploying your chalice application, received the following error:

An error occurred (DeleteConflict) when calling the DeleteRole operation:
Cannot delete entity, must detach all policies first.

```
I tried to both specify the json for the permission `and` to have those permission already configured in AWS. Finally, it is unclear if permission should be in `policy.json` or `policy-dev.json`
Here is my config.json
```
{
"version": "2.0",
"app_name": "top200",
"stages": {
"dev": {
"api_gateway_stage": "top200",
"manage_iam_role": false,
"autogen_policy": false,
"iam_role_arn": "arn:aws:iam::439359573308:role/top200-dev"
}
}
}
```
Here is my policy.json
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:eu-west-1:439359573308:dbuser:arn:aws:rds:us-east-1:439359573308:db:top200-portal/top200admin"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"*"
],
"Sid": "5a1a5b736299483abb776aeee43f4a88"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}
```

Contributor guide

Open the contributing guide

Research direction

Start with chalice/deploy/deployer.py and chalice/awsclient.py around role deletion, then inspect how manage_iam_role, autogen_policy, and iam_role_arn are handled for stages. Reproduce the deployment error and check the handling of policy.json versus policy-dev.json; done when deployment uses the intended role without the DeleteConflict error.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, mysql, python
Domain
backend, cloud, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.