aws / aws/amazon-sagemaker-examples
GPT-4ALL-J on Inferentia2
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
**Link to the notebook**
https://github.com/aws/amazon-sagemaker-examples/blob/main/inference/generativeai/llm-workshop/lab8-Inferentia2-gpt4all-j/inferentia2-llm-GPT4allJ.ipynb
**Describe the bug**
Keep getting the following warning:
- Your model is not compiled. Please compile your model before using Inferentia.
and this error:
- ClientError: An error occurred (ValidationException) when calling the CreateEndpointConfig operation: Instance type ml.inf2.8xlarge is not supported for the chosen region. Please choose some other instance type.
**To reproduce**
Same steps as stated in notebook
I have already set my region to us-east-1 for all services, yet I still encounter the error for Inferentia2 instance
Contributor guide
Research direction
Start with the linked inferentia2-llm-GPT4allJ.ipynb and inspect the deployment cells that compile the model and create the endpoint configuration. Reproduce the reported warning and ValidationException in us-east-1; done means the notebook’s Inferentia2 deployment completes without these errors or clearly identifies the unsupported configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, jupyter-notebook
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100