aws / aws/amazon-sagemaker-examples

[Bug Report]

Open
#3,931 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/nlp/realtime/llm/opt30b/djl_deepspeed_deploy_opt30b.ipynb

**Describe the bug**
when loading the model i get this error:
` The model 'InferenceEngine' is not supported for text-generation `

**To reproduce**
A clear, step-by-step set of instructions to reproduce the bug.

**Logs**
If applicable, add logs to help explain your problem.
You may also attach an `.ipynb` file to this issue if it includes relevant logs or output.

Contributor guide

Open the contributing guide

Research direction

Start with inference/nlp/realtime/llm/opt30b/djl_deepspeed_deploy_opt30b.ipynb and run the model-loading step to confirm the reported InferenceEngine text-generation error. The issue does not provide reproduction steps or logs, so determine the failing notebook stage and collect the missing output; done means the cause and a supported resolution or limitation are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, jupyter-notebook
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.