Not able to deploy Gemma3 model using TGI container

I am trying to deploy Gemma 3 4B model using TGI container in the below notebook and running into issue where end point is not created.
https://github.com/aws-samples/sagemaker-genai-hosting-examples/blob/main/Gemma3/Gemma-3-4B-instruct-TGI.ipynb

Step to reproduce:
1. Run Gemma-3-4B-instruct-TGI.ipynb notebook
2. Follow steps out lined in the notebook.
3. Confirmed I am using HF token and I have access to this Gated model.

Error
```
│ ╰──────────────────────────────────────────────────────────────────────────╯ │

2025-05-08T15:46:38.762Z

│ │

2025-05-08T15:46:38.762Z

│ /usr/src/server/text_generation_server/utils/hub.py:111 in weight_hub_files │

2025-05-08T15:46:38.762Z

│ │

2025-05-08T15:46:38.762Z

│ 108 │ │ filenames = _weight_hub_files_from_model_info(info, extension) │

2025-05-08T15:46:38.762Z

│ 109 │ │

2025-05-08T15:46:38.762Z

│ 110 │ if not filenames: │

2025-05-08T15:46:38.762Z

│ ❱ 111 │ │ raise EntryNotFoundError( │

2025-05-08T15:46:38.762Z

│ 112 │ │ │ f"No {extension} weights found for model {model_id} and re │

2025-05-08T15:46:38.762Z

│ 113 │ │ │ None, │

2025-05-08T15:46:38.762Z

│ 114 │ │ ) │

2025-05-08T15:46:38.762Z

│ │

2025-05-08T15:46:38.762Z

│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │

2025-05-08T15:46:38.762Z

│ │ api = │ │

2025-05-08T15:46:38.762Z

│ │ extension = '.bin' │ │

2025-05-08T15:46:38.762Z

│ │ filenames = [] │ │

2025-05-08T15:46:38.762Z

│ │ info = ModelInfo( │ │

2025-05-08T15:46:38.762Z

│ │ │ id='google/gemma-3-4b-it', │ │

2025-05-08T15:46:38.762Z

│ │ │ author='google', │ │

2025-05-08T15:46:38.762Z

│ │ │ sha='093f9f388b31de276ce2de164bdc2081324b9767', │ │

2025-05-08T15:46:38.762Z

│ │ │ created_at=datetime.datetime(2025, 2, 20, 21, 20, 7, │ │

2025-05-08T15:46:38.762Z

│ │ tzinfo=datetime.timezone.utc), │ │

2025-05-08T15:46:38.762Z

│ │ │ last_modified=datetime.datetime(2025, 3, 21, 20, 20, 53, │ │

2025-05-08T15:46:38.762Z

│ │ tzinfo=datetime.timezone.utc), │ │

2025-05-08T15:46:38.762Z

│ │ │ private=False, │ │

2025-05-08T15:46:38.762Z

│ │ │ disabled=False, │ │

2025-05-08T15:46:38.762Z

│ │ │ downloads=611914, │ │

2025-05-08T15:46:38.762Z

│ │ │ downloads_all_time=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ gated='manual', │ │

2025-05-08T15:46:38.762Z

│ │ │ gguf=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ inference=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ inference_provider_mapping=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ likes=501, │ │

2025-05-08T15:46:38.762Z

│ │ │ library_name='transformers', │ │

2025-05-08T15:46:38.762Z

│ │ │ tags=[ │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'transformers', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'safetensors', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'gemma3', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'image-text-to-text', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'conversational', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'arxiv:1905.07830', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'arxiv:1905.10044', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'arxiv:1911.11641', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'arxiv:1904.09728', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'arxiv:1705.03551', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ... +29 │ │

2025-05-08T15:46:38.762Z

│ │ │ ], │ │

2025-05-08T15:46:38.762Z

│ │ │ pipeline_tag='image-text-to-text', │ │

2025-05-08T15:46:38.762Z

│ │ │ mask_token=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ card_data={'base_model': 'google/gemma-3-4b-pt', │ │

2025-05-08T15:46:38.762Z

│ │ 'datasets': None, 'eval_results': None, 'language': None, │ │

2025-05-08T15:46:38.762Z

│ │ 'library_name': 'transformers', 'license': 'gemma', │ │

2025-05-08T15:46:38.762Z

│ │ 'license_name': None, 'license_link': None, 'metrics': None, │ │

2025-05-08T15:46:38.762Z

│ │ 'model_name': None, 'pipeline_tag': 'image-text-to-text', │ │

2025-05-08T15:46:38.762Z

│ │ 'tags': None, 'extra_gated_heading': 'Access Gemma on │ │

2025-05-08T15:46:38.762Z

│ │ Hugging Face', 'extra_gated_prompt': 'To access Gemma on │ │

2025-05-08T15:46:38.762Z

│ │ Hugging Face, you’re required to review and agree to │ │

2025-05-08T15:46:38.762Z

│ │ Google’s usage license. To do this, please ensure you’re │ │

2025-05-08T15:46:38.762Z

│ │ logged in to Hugging Face and click below. Requests are │ │

2025-05-08T15:46:38.762Z

│ │ processed immediately.', 'extra_gated_button_content': │ │

2025-05-08T15:46:38.762Z

│ │ 'Acknowledge license'}, │ │

2025-05-08T15:46:38.762Z

│ │ │ widget_data=[ │ │

2025-05-08T15:46:38.762Z

│ │ │ │ {'text': 'Hi, what can you help me with?'}, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ {'text': 'What is 84 * 3 / 2?'}, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ { │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'text': 'Tell me an interesting fact about the │ │

2025-05-08T15:46:38.762Z

│ │ universe!' │ │

2025-05-08T15:46:38.762Z

│ │ │ │ }, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ { │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'text': 'Explain quantum computing in simple │ │

2025-05-08T15:46:38.762Z

│ │ terms.' │ │

2025-05-08T15:46:38.762Z

│ │ │ │ } │ │

2025-05-08T15:46:38.762Z

│ │ │ ], │ │

2025-05-08T15:46:38.762Z

│ │ │ model_index=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ config={ │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'architectures': [ │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'Gemma3ForConditionalGeneration' │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ], │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'model_type': 'gemma3', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'processor_config': { │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'chat_template': "{{ bos_token }}\n{%- if │ │

2025-05-08T15:46:38.762Z

│ │ messages[0]['role'] == 'system' -%}\n {%- if │ │

2025-05-08T15:46:38.762Z

│ │ messages[0"+1452 │ │

2025-05-08T15:46:38.762Z

│ │ │ │ }, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'tokenizer_config': { │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'bos_token': '', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'chat_template': "{{ bos_token }}\n{%- if │ │

2025-05-08T15:46:38.762Z

│ │ messages[0]['role'] == 'system' -%}\n {%- if │ │

2025-05-08T15:46:38.762Z

│ │ messages[0"+1452, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'eos_token': '', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'pad_token': '', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'unk_token': '', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ 'use_default_system_prompt': False │ │

2025-05-08T15:46:38.762Z

│ │ │ │ } │ │

2025-05-08T15:46:38.762Z

│ │ │ }, │ │

2025-05-08T15:46:38.762Z

│ │ │ transformers_info=TransformersInfo( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ auto_model='AutoModelForImageTextToText', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ custom_class=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ pipeline_tag='image-text-to-text', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ processor='AutoProcessor' │ │

2025-05-08T15:46:38.762Z

│ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ trending_score=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ siblings=[ │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='.gitattributes', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='README.md', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='added_tokens.json', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='chat_template.json', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='config.json', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='generation_config.json', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='model-00001-of-00002.safetensors', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='model-00002-of-00002.safetensors', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='model.safetensors.index.json', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ RepoSibling( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ rfilename='preprocessor_config.json', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ size=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ blob_id=None, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ │ lfs=None │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ... +5 │ │

2025-05-08T15:46:38.762Z

│ │ │ ], │ │

2025-05-08T15:46:38.762Z

│ │ │ spaces=[ │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'hadadrjt/ai', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'TIGER-Lab/MEGA-Bench', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'whitecircle-ai/circle-guard-bench', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'VIDraft/Gemma-3-R1984-4B', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'merterbak/gemma-3', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'CalvinU/writing-prototypes', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'Tomoniai/gemma-3-chat', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'hanzla/PlaygroundGemma3', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'AC2513/gemma-translate', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ 'stiv14/pdf-multilanguage-qa-role', │ │

2025-05-08T15:46:38.762Z

│ │ │ │ ... +46 │ │

2025-05-08T15:46:38.762Z

│ │ │ ], │ │

2025-05-08T15:46:38.762Z

│ │ │ safetensors=SafeTensorsInfo( │ │

2025-05-08T15:46:38.762Z

│ │ │ │ parameters={'BF16': 4300079472}, │ │

2025-05-08T15:46:38.762Z

│ │ │ │ total=4300079472 │ │

2025-05-08T15:46:38.762Z

│ │ │ ), │ │

2025-05-08T15:46:38.762Z

│ │ │ security_repo_status=None │ │

2025-05-08T15:46:38.762Z

│ │ ) │ │

2025-05-08T15:46:38.762Z

│ │ model_id = 'google/gemma-3-4b-it' │ │

2025-05-08T15:46:38.762Z

│ │ revision = None │ │

2025-05-08T15:46:38.762Z

│ ╰──────────────────────────────────────────────────────────────────────────╯ │

2025-05-08T15:46:38.762Z

╰──────────────────────────────────────────────────────────────────────────────╯

2025-05-08T15:46:38.762Z

EntryNotFoundError: No .bin weights found for model google/gemma-3-4b-it and

2025-05-08T15:46:39.262Z

revision None.
```

Start with Gemma-3-4B-instruct-TGI.ipynb and reproduce the deployment using the stated Hugging Face token and model access. Inspect the TGI container error at /usr/src/server/text_generation_server/utils/hub.py:111, where no .bin weights are found despite safetensors files being listed. Done means the notebook creates the SageMaker endpoint successfully.

Written by the indexing model from the issue text.