crashappsec / crashappsec/chalk
Restrict the S3 URI to only refer to folders and allow users to control object names
- Dominant language
- Nim
- Stars
- 436
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
## Description
When configuring the `AWS_S3_BUCKET_URI` for the S3 sink, there is an implicit convention used by chalk on the naming scheme, which is not necessarily obvious to the user unless they [take a close look at the con4m files](https://github.com/crashappsec/chalk/blob/f740ac6266a589a8ac6dfcf5c41dbb97ea1a3cfe/src/configs/base_sinks.c4m#L106-L112).
In particular, if a user sets `AWS_S3_BUCKET_URI` to `s3://chalk-test/chalktest/foo.json` the objects will be getting created in the format `s3://chalk-test/chalktest/1706138848344-T9QNK53FX8GMWYCNCDJSDV2PK0-foo.json`. Notice how foo.json is the suffix of that file.
Even if a folder already exists, if a trailing slash is omitted, then an object on the same directory as the folder will be getting created. For instance, if one sets `export AWS_S3_BUCKET_URI=s3://chalk-test/chalktest`, then `s3://chalk-test/1706138621398-08T8N85FVBC0V5XZ40X812GRZC-chalktest` will be getting created.
### Proposal for Paths
Ideally we would want consistent behavior regardless of whether a trailing slash is there or not by the user. The fact that s3 could have the same name for an object and folder at the same level of the hierarchy is not known to all users and its likely that a user accidentally omits a `/` when defining a path. To this end, we can instead use `AWS_S3_FOLDER` to explicitly require a folder to be used for all chalkmarks to be inserted there. If such a folder does not exist, then chalk could automatically create it or yield an error if it fails to do so.
### Proposal for object names
Chalk takes an opinionated stance inserting a random prefix to [ensure uniqueness](https://github.com/crashappsec/chalk/blob/f740ac6266a589a8ac6dfcf5c41dbb97ea1a3cfe/src/configs/base_sinks.c4m#L106-L112) of generated objects. However this might not be intended behavior for users. For instance, it could be that users want a fixed name to always be used for all their reports, with each report overwriting their previous ones, or they might want to dynamically name their objects from some keys of interest (e.g., hostname, action etc.). The proposal would be to define some keys that can be used to set an `awsS3ObjectName` field via the configuration. For instance, one could define n fields and a random prefix to be used to name an object, optionally generate a hash of all the fields passed in as the fingerprint of the object etc.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the S3 sink configuration in src/configs/base_sinks.c4m and trace how AWS_S3_BUCKET_URI is interpreted and how object names are generated. Compare the current trailing-slash behavior with the proposed explicit folder and configurable object-name behavior. Done means the configuration semantics are defined and implemented with coverage for folder paths and user-controlled names.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100