aws / aws/aws-cdk

(aws-glue): Glue Job should support zipping directories when used for using extra python files

Open
#21,951 3 comments 2 reactions 0 assignees View on GitHub
@aws-cdk/aws-glue effort/small feature-request p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
71

Description

### Describe the feature

One can zip a directory and then send that file as an "extra python files" entry.

### Use Case

E.g. I want my `./glue/business_logic` folder (which contains an `__init__.py` and several `*.py` files and even sub-packages) automatically zipped so it is reachable in glue code as `from business_logic.specific import whatever`.

### Proposed Solution

remove the test for a single file from the [AssetCode constructor](https://github.com/humanzz/aws-cdk/blob/main/packages/@aws-cdk/aws-glue/lib/code.ts#L66-L72):

```ts
constructor(private readonly path: string, private readonly options: s3assets.AssetOptions = { }) {
super();

if (fs.lstatSync(this.path).isDirectory()) {
throw new Error(`Code path ${this.path} is a directory. Only files are supported`);
}
}
```

### Other Information

_No response_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

2.40.0 (build 56ba2ab)

### Environment details (OS name and version, etc.)

mac

Contributor guide

Open the contributing guide

Research direction

Start at packages/@aws-cdk/aws-glue/lib/code.ts, especially the AssetCode constructor and its directory check. Verify how an extra Python files path is packaged; done means a directory containing Python modules is accepted and zipped for use by AWS Glue.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, data-engineering
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.