aws / aws/sagemaker-python-sdk
Unable to increase context length in SM serverless customizations
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 1.3k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 35
Description
**Describe the feature you'd like**
Currently customers cannot use GRPO for Qwen model customization for models using long context (more than 2048 tokens)
**How would this feature be used? Please describe.**
Should work similar to JumpStart training implementation:
estimator = JumpStartEstimator(
model_id="meta-textgeneration-llama-2-7b",
hyperparameters={
"max_input_length": "4096", # Update context length here
"max_total_tokens": "4096"
}
)
**Describe alternatives you've considered**
Using SM training jobs for GRPO training with recipes.
**Additional context**
Customer: Intuit (a high-priority issue blocking usage of SM serverless customization service).
Contributor guide
Research direction
Start with the serverless customization path and compare its configuration with the JumpStartEstimator example in the issue. Trace how GRPO customization handles Qwen models and the max_input_length and max_total_tokens hyperparameters. Done means customers can use those settings for context lengths above 2048 tokens in SM serverless customization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100