Error: DeepSpeed Zero-3 is not compatible with `low_cpu_mem_usage=True` or with passing a `device_map`

### Product Version

- [ ] Amazon SageMaker Studio Classic
- [X] Amazon SageMaker Studio
- [ ] Issue is not related to SageMaker Studio

### Issue Description

I was using SageMaker Studio to domain-train a model (base model: huggingface-llm-mistral-7b) using a `ml.g5.24xlarge` instance. I left all values at default other than pointing it to specific buckets for the training data and to output the trained model and adjusted the hyperparameters with:
- Perf type: lora
- Instruction-Train The Model: False
- Epochs: 3

At just over an hour (3,909 seconds) into the training run, I received the error:

```
AlgorithmError: ExecuteUserScriptError: ExitCode 1 ErrorMessage "raise ValueError( ValueError DeepSpeed Zero-3 is not compatible with `low_cpu_mem_usage=True` or with passing a `device_map`. ERROR:root:Subprocess script failed with return code: 1 Traceback (most recent call last) File "/opt/conda/lib/python3.10/site-packages/sagemaker_jumpstart_script_utilities/subprocess.py", line 9, in run_with_error_handling subprocess.run(command, shell=shell, check=True) File "/opt/conda/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['deepspeed', '--num_gpus=4', '/opt/conda/lib/python3.10/site-packages/sagemaker_jumpstart_huggingface_script_utilities/fine_tuning/run_clm.py', '--deepspeed', 'ds_config.json', '--model_name_or_path', '/tmp', '--train_file', '/opt/ml/input/data/training', '--do_train', '--output_dir', '/opt/ml/model', '--num_train_epochs', '3', '--gradient_accumulation_steps', '8', '--per_device_train_batch_siz
```

I came across [this specific post](https://github.com/huggingface/peft/issues/306#issuecomment-1508367601), but don't believe these to be values I can adjust via SageMaker Studio.

Any thoughts on this?

### Expected Behavior

Expected the model to be domain-trained successfully.

### Observed Behavior

Observed the error identified in the Issue Description section.

### Product Category

JumpStart

### Feedback Category

Reliability and Stability

### Other Details

_No response_

Start with the reported deepspeed command, the ds_config.json argument, and the referenced run_clm.py entry point to determine where the incompatible settings are introduced in SageMaker Studio. Reproduce the domain-training run and consider the issue complete when the model trains successfully without the reported DeepSpeed Zero-3 error.

Written by the indexing model from the issue text.