aws-samples / aws-samples/sagemaker-custom-project-templates
[Feature] A 'feature pipeline' template
- Dominant language
- Python
- Stars
- 237
- Forks
- 170
- PR merge metrics
- No merged PRs in 30d
Description
For my use case, we're looking for a deployable project template/stack which:
- Creates & updates a SageMaker Feature Store Feature Group through the CI/CD pipeline, based on a configuration file, and
- Creates and runs a sample SageMaker Pipeline to transform raw data (can assume S3 source) and ingest it into the Feature Group
We started from the SageMaker built-in 'model building and training' template, which already helps a lot with the SageMaker Pipeline / processing jobs aspect... But as far as I can tell there isn't one that covers managing a feature group (schema, tags, feature-level metadata) via CI/CD which is a shame.
I've put some initial thought into what a transferable sample could look like, but haven't had time yet to draft up an attempt!
## Design ideas
### Feature Group management
1. As of today, CloudFormation [AWS::SageMaker::FeatureGroup](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sagemaker-featuregroup.html) gives a native mechanism to create/update/delete Feature Groups, but as far as I can tell it doesn't support [feature-level metadata](https://aws.amazon.com/about-aws/whats-new/2022/07/amazon-sagemaker-feature-supports-metadata-search/) (parameters and descriptions) which are important for enterprise usage.
2. I suspect different organizations might wish to apply different levels of safety checks to the CloudFormation default, for actions that might result in deleting and replacing the Feature Group (I assume CFn wouldn't do automatic data replication in this case?)
For these two reasons, I was leaning towards defining a custom feature group+feature metadata configuration e.g. in JSON/YAML, and using custom Python code in CodeBuild to reconcile the current vs target feature group config and either perform the necessary updates or fail.
To avoid trying to overload parameters of the Service Catalog template itself, I was thinking such a template could take only basic parameters like the feature group name, ID field name and type, etc... Not creating any actual features until the initial 'seed code' config file is updated to add features in (since adding features to a group is a supported operation but removing them is not).
### Feature transformation and ingestion
In our environment we're currently using two-step SageMaker Pipelines for feature engineering: One processing job to extract and transform raw data (from S3), and a separate one to do the feature store ingestion - to keep this component as easily re-usable as possible and separately right-size infrastructure. We've so far experimented with both PySparkProcessor and Data Wrangler for these steps, so that general architecture was my assumed pattern for the actual feature ingestion pipeline.
### One or many feature groups?
I guess there's probably no harm in such a template theoretically supporting multiple pipelines, multiple feature groups - just like the model building & training template currently supports multiple pipelines... But our planned process is pretty much to deploy a separate project template per feature group at this stage.
Contributor guide
Research direction
Start by reviewing the existing SageMaker built-in model building and training template, then compare its SageMaker Pipeline and processing-job structure with the requested Feature Group management flow. The issue mentions CloudFormation, CodeBuild, JSON/YAML configuration, S3 input, and separate transformation and ingestion steps; done would be a deployable project template covering those requirements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- ci-cd, cloud, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100