[glue] Support Glue::Crawler as L2 construct
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 74
Description
We are getting more heavily invested into AWS Glue. Right now we have some L2 constructs internally and it would be good to have them upstream.
### Use Case
Have L2 Construct for L1 `CfnCrawler`.
### Proposed Solution
Have a workflow like this:
```typescript
const bucket = new s3.Bucket(stack, 'bucket');
new glue.Crawler(stack, 'crawler', {
targets: [
new S3Target(bucket, {exclusions: ["abc"]})
]
});
```
**Here is a test, that we run for additional info**:
```typescript
test('Create Crawler', () => {
// GIVEN
const app = new cdk.App();
const stack = new cdk.Stack(app, 'stack');
const bucket = new s3.Bucket(stack, 'bucket', {bucketName: "test"});
// WHEN
new glue.Crawler(stack, 'crawler', {
targets: [new S3Target(bucket, {exclusions: ["abc"]})]
});
// THEN
cdkExpect(stack).to(haveResource('AWS::Glue::Crawler',
{
Type: 'AWS::Glue::Crawler',
Properties: {
Role: { "Fn::GetAtt": ["crawlerServiceRole63A6FAF9", "Arn"]},
Targets:{
S3Targets: [{
Path: { "Fn::Join": [ "", [ "s3://", {Ref: "bucket43879C71"}]]},
Exclusions: ["abc"]
}]
},
Configuration: "{\"Version\":1,\"CrawlerOutput\":{\"Partitions\":{\"AddOrUpdateBehavior\":\"InheritFromTable\"},\"Tables\":{\"AddOrUpdateBehavior\":\"MergeNewColumns\"}},\"Grouping\":{\"TableGroupingPolicy\":\"CombineCompatibleSchemas\"}}"
}
}, ResourcePart.CompleteDefinition));
cdkExpect(stack).to(haveResourceLike('AWS::IAM::Role',
{
Type: 'AWS::IAM::Role',
Properties: {
AssumeRolePolicyDocument: {
Statement: [
{
Action: "sts:AssumeRole",
Effect: "Allow",
Principal: {Service: "glue.amazonaws.com"}
}
]},
ManagedPolicyArns: [
{
"Fn::Join": [
"", [
"arn:",
{Ref: "AWS::Partition"},
":iam::aws:policy/service-role/AWSGlueServiceRole"
]
]
}
],
}
}, ResourcePart.CompleteDefinition));
cdkExpect(stack).to(haveResourceLike('AWS::IAM::Policy',
{ PolicyDocument: {
Statement: [
{
Resource: [{
"Fn::GetAtt": ["bucket43879C71", "Arn"]
}]
}
]},
Roles: [
{ Ref: "crawlerServiceRole63A6FAF9" }
]
}
));
});
```
### Other
* [X] :wave: I may be able to implement this feature request
* [ ] :warning: This feature might incur a breaking change
---
This is a :rocket: Feature Request
Contributor guide
Research direction
Start by reading the L1 CfnCrawler API and the TypeScript example and test included in the issue. The work is done when a glue.Crawler and S3Target construct can produce the shown AWS::Glue::Crawler resource, service role, and IAM policy assertions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100