aws / aws/amazon-sagemaker-examples
[Bug Report]
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/nlp/realtime/llm/opt30b/djl_deepspeed_deploy_opt30b.ipynb
**Describe the bug**
when loading the model i get this error:
` The model 'InferenceEngine' is not supported for text-generation `
**To reproduce**
A clear, step-by-step set of instructions to reproduce the bug.
**Logs**
If applicable, add logs to help explain your problem.
You may also attach an `.ipynb` file to this issue if it includes relevant logs or output.
Contributor guide
Research direction
Start with inference/nlp/realtime/llm/opt30b/djl_deepspeed_deploy_opt30b.ipynb and run the model-loading step to confirm the reported InferenceEngine text-generation error. The issue does not provide reproduction steps or logs, so determine the failing notebook stage and collect the missing output; done means the cause and a supported resolution or limitation are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100