aws / aws/aws-parallelcluster

Bug: CLI crashes with "'str' object has no attribute 'get'" when updating SharedStorage MountDir

Open
#7,162 1 comment 0 reactions 0 assignees View on GitHub
3.x Backlog Bug
Dominant language
Python
Stars
888
Forks
314
Avg merge
1d 10h
Merged PRs (30d)
43

Description

# Bug: CLI crashes with "'str' object has no attribute 'get'" when updating SharedStorage MountDir

## Required Info

- **AWS ParallelCluster version:** 3.14.0
- **Cluster name:** (any cluster with SharedStorage)
- **Arn of the cluster CloudFormation main stack:** N/A (reproducible on any cluster)
- **Output of `pcluster describe-cluster`**:

```json
{
"version": "3.14.0",
"clusterStatus": "CREATE_COMPLETE",
"computeFleetStatus": "RUNNING",
"scheduler": {
"type": "slurm"
}
}
```

### Full cluster configuration (minimal reproduction):

```yaml
Region: eu-north-1
Image:
Os: rhel9
HeadNode:
Imds:
Secured: true
InstanceType: c6i.large
Networking:
SubnetId: subnet-02022469c3451395b
AdditionalSecurityGroups:
- sg-0b6398ec0dbba2ef2
ElasticIp: true
Ssh:
KeyName: my_key

Scheduling:
Scheduler: slurm
SlurmQueues:
- Name: hpc6a
Networking:
SubnetIds:
- subnet-0f51e28e2cd594d96
AdditionalSecurityGroups:
- sg-0b6398ec0dbba2ef2
CapacityType: ONDEMAND
ComputeSettings:
LocalStorage:
RootVolume:
Size: 200
VolumeType: gp3
ComputeResources:
- Name: hpc6a-48xlarge
InstanceType: hpc6a.48xlarge
MinCount: 0
MaxCount: 8
DisableSimultaneousMultithreading: true
Efa:
Enabled: true
Networking:
PlacementGroup:
Enabled: true

SharedStorage:
- Name: fsx-lustre
MountDir: /fsx
StorageType: FsxLustre
FsxLustreSettings:
StorageCapacity: 1200
Imds:
ImdsSupport: v2.0
```

## Bug description and how to reproduce

### Description

When running `pcluster update-cluster` with a configuration that changes the `MountDir` of a SharedStorage item, the CLI crashes with an unhandled exception instead of returning a proper validation error.

**Error message:**
```json
{
"message": "Cluster update failed.\n'str' object has no attribute 'get'"
}
```

### Steps to reproduce

1. Create a cluster with SharedStorage that has a `MountDir`:
```yaml
SharedStorage:
- Name: fsx-lustre
MountDir: /fsx
StorageType: FsxLustre
FsxLustreSettings:
StorageCapacity: 1200
```

2. Update the cluster with a changed `MountDir` (no other changes required):
```yaml
SharedStorage:
- Name: fsx-lustre
MountDir: /fsx1 # Changed from /fsx
StorageType: FsxLustre
FsxLustreSettings:
StorageCapacity: 1200
```

3. Run:
```bash
pcluster update-cluster --cluster-name --region --cluster-configuration
```

4. CLI crashes with `'str' object has no attribute 'get'`

### Conditions

The bug occurs when **all three conditions** are met:
1. Cluster compute fleet status is anything other than `STOPPED` (e.g., `RUNNING`, `UNKNOWN`, `STARTING`, etc.)
2. SharedStorage `MountDir` is changed
3. Slurm scheduler is used

### Workaround

Stop the compute fleet before updating `MountDir`:

```bash
pcluster update-compute-fleet --cluster-name --region --status STOP_REQUESTED
# Wait for status to become STOPPED
pcluster describe-compute-fleet --cluster-name --region
# Then update succeeds
pcluster update-cluster --cluster-name --region --cluster-configuration
```

```log
(.pcluster3140) ➜ pcluster update-cluster --cluster-name repro-cluster --cluster-configuration repro-cluster.yaml --debug
{
"message": "Cluster update failed.\n'str' object has no attribute 'get'"
}

(.pcluster3140) ➜ pcluster update-compute-fleet --cluster-name repro-cluster --region eu-north-1 --status STOP_REQUESTED
{
"status": "STOP_REQUESTED",
"lastStatusUpdatedTime": "2025-12-19T01:24:42.850Z"
}

(.pcluster3140) ➜ pcluster describe-compute-fleet --cluster-name repro-cluster --region eu-north-1
{
"status": "STOPPED",
"lastStatusUpdatedTime": "2025-12-19T01:25:02.699Z"
}

(.pcluster3140) ➜ pcluster update-cluster --cluster-name repro-cluster --cluster-configuration repro-cluster.yaml --debug
{
"cluster": {
"clusterName": "repro-cluster",
"cloudformationStackStatus": "UPDATE_IN_PROGRESS",
"cloudformationStackArn": "arn:aws:cloudformation:eu-north-1:REDACTED:stack/repro-cluster/42742080-dc70-11f0-9148-06a50478435d",
"region": "eu-north-1",
"version": "3.14.0",
"clusterStatus": "UPDATE_IN_PROGRESS",
"scheduler": {
"type": "slurm"
}
},
"validationMessages": [
{
"level": "INFO",
"type": "DeletionPolicyValidator",
"message": "The DeletionPolicy is set to Delete. The storage 'fsx-lustre' will be deleted when you remove it from the configuration when performing a cluster update or deleting the cluster."
}
],
"changeSet": [
{
"parameter": "SharedStorage[fsx-lustre].MountDir",
"requestedValue": "/fsx1",
"currentValue": "/fsx"
}
]
}

```

## Root Cause Analysis

### Bug Location

**File:** `cli/src/pcluster/config/update_policy_utils.py`
**Line:** 25
**Class:** `SharedStorageChangeInfo.__init__()`

### Traceback

```log
2025-12-18 17:14:42,145 - INFO - entrypoint.py:260:run() - Handling CLI command update-cluster
2025-12-18 17:14:42,146 - DEBUG - entrypoint.py:261:run() - Parsed CLI arguments: args(REDACTED), extra_args([])
2025-12-18 17:14:42,147 - DEBUG - util.py:170:_assert_node_executable() - Found Node.js executable in /Users/REDACTED/.nvm/versions/node/v20.19.0/bin/node
2025-12-18 17:14:42,208 - DEBUG - util.py:191:_assert_node_version() - Found Node.js version (v20.19.0
)
2025-12-18 17:14:42,210 - INFO - common.py:44:_set_region() - Setting AWS Region to eu-north-1
2025-12-18 17:14:43,057 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=cloudformation, operation=DescribeStacks, params={'StackName': 'repro-cluster'}
2025-12-18 17:14:43,764 - INFO - import_cdk.py:10:import_cdk() - Start importing
2025-12-18 17:14:43,769 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=cloudformation, operation=DescribeStacks, params={'StackName': 'repro-cluster'}
2025-12-18 17:14:44,006 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeInstances, params={'Filters': [{'Name': 'tag:parallelcluster:cluster-name', 'Values': ['repro-cluster']}, {'Name': 'instance-state-name', 'Values': ['pending', 'running', 'stopping', 'stopped']}, {'Name': 'tag:parallelcluster:node-type', 'Values': ['HeadNode']}]}
2025-12-18 17:14:44,766 - INFO - cluster.py:488:_validate_and_parse_config() - Validating cluster configuration...
2025-12-18 17:14:44,787 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=cloudformation, operation=DescribeStackResources, params={'StackName': 'repro-cluster'}
2025-12-18 17:14:45,277 - DEBUG - common.py:190:_validator_execute() - Executing validator OsCustomAmiValidator
2025-12-18 17:14:45,277 - DEBUG - common.py:190:_validator_execute() - Executing validator SecurityGroupsValidator
2025-12-18 17:14:45,277 - DEBUG - common.py:190:_validator_execute() - Executing validator SecurityGroupsValidator
2025-12-18 17:14:45,373 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeSecurityGroups, params={'GroupIds': ['sg-0b6398ec0dbba2ef2']}
2025-12-18 17:14:45,595 - DEBUG - common.py:190:_validator_execute() - Executing validator ElasticIpValidator
2025-12-18 17:14:45,595 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeThroughputValidator
2025-12-18 17:14:45,595 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeThroughputIopsValidator
2025-12-18 17:14:45,595 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceTypeValidator
2025-12-18 17:14:45,600 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeInstanceTypeOfferings, params={}
2025-12-18 17:14:47,093 - DEBUG - common.py:190:_validator_execute() - Executing validator SharedStorageEfsSettingsValidator
2025-12-18 17:14:47,093 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeThroughputValidator
2025-12-18 17:14:47,093 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeThroughputIopsValidator
2025-12-18 17:14:47,093 - DEBUG - common.py:190:_validator_execute() - Executing validator PlacementGroupNamingValidator
2025-12-18 17:14:47,098 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeInstanceTypes, params={'InstanceTypes': ['hpc6a.48xlarge']}
2025-12-18 17:14:47,301 - DEBUG - common.py:190:_validator_execute() - Executing validator NameValidator
2025-12-18 17:14:47,301 - DEBUG - common.py:190:_validator_execute() - Executing validator SchedulableMemoryValidator
2025-12-18 17:14:47,302 - DEBUG - common.py:190:_validator_execute() - Executing validator PlacementGroupNamingValidator
2025-12-18 17:14:47,302 - DEBUG - common.py:190:_validator_execute() - Executing validator SecurityGroupsValidator
2025-12-18 17:14:47,302 - DEBUG - common.py:190:_validator_execute() - Executing validator SecurityGroupsValidator
2025-12-18 17:14:47,308 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeSubnets, params={'SubnetIds': ['subnet-0f51e28e2cd594d96']}
2025-12-18 17:14:47,528 - DEBUG - common.py:190:_validator_execute() - Executing validator NameValidator
2025-12-18 17:14:47,528 - DEBUG - common.py:190:_validator_execute() - Executing validator EfaMultiAzValidator
2025-12-18 17:14:47,528 - DEBUG - common.py:190:_validator_execute() - Executing validator EfaValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator MultiAzPlacementGroupValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator DuplicateNameValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator MaxCountValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator QueueSubnetsValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator SlurmNodePrioritiesWarningValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator SingleInstanceTypeSubnetValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator EfaSecurityGroupValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator EfaPlacementGroupValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator ComputeResourceSizeValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator CapacityTypeValidator
2025-12-18 17:14:47,529 - DEBUG - common.py:190:_validator_execute() - Executing validator FeatureRegionValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator ExternalSlurmdbdVsDatabaseIncompatibility
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator ExternalSlurmdbdRequiresCustomMungeKey
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator ExternalSlurmdbdTrafficNotEncrypted
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator DuplicateNameValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator RootVolumeEncryptionConsistencyValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator MaxCountValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator MaxCountValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator SharedStorageNameValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxDraValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxS3Validator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxPersistentOptionsValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxBackupOptionsValidator
2025-12-18 17:14:47,536 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxStorageTypeOptionsValidator
2025-12-18 17:14:47,537 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxStorageCapacityValidator
2025-12-18 17:14:47,537 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxBackupIdValidator
2025-12-18 17:14:47,537 - DEBUG - common.py:190:_validator_execute() - Executing validator DeletionPolicyValidator
2025-12-18 17:14:47,537 - DEBUG - common.py:190:_validator_execute() - Executing validator LogRotationValidator
2025-12-18 17:14:47,537 - DEBUG - common.py:190:_validator_execute() - Executing validator DetailedMonitoringValidator
2025-12-18 17:14:47,547 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeInstanceTypes, params={'InstanceTypes': ['c6i.large']}
2025-12-18 17:14:47,752 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeSubnets, params={'SubnetIds': ['subnet-02022469c3451395b']}
2025-12-18 17:14:47,926 - INFO - import_cdk.py:16:import_cdk() - Import complete in 4 seconds
2025-12-18 17:14:47,971 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeImages, params={'ImageIds': ['ami-0aeb582419dd31854']}
2025-12-18 17:14:48,198 - DEBUG - common.py:190:_validator_execute() - Executing validator RegionValidator
2025-12-18 17:14:48,198 - DEBUG - common.py:190:_validator_execute() - Executing validator ClusterNameValidator
2025-12-18 17:14:48,198 - DEBUG - common.py:190:_validator_execute() - Executing validator ArchitectureOsValidator
2025-12-18 17:14:48,198 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceTypeBaseAMICompatibleValidator
2025-12-18 17:14:48,198 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceTypeOSCompatibleValidator
2025-12-18 17:14:48,199 - DEBUG - common.py:190:_validator_execute() - Executing validator DuplicateNameValidator
2025-12-18 17:14:48,199 - DEBUG - common.py:190:_validator_execute() - Executing validator DuplicateNameValidator
2025-12-18 17:14:48,199 - DEBUG - common.py:190:_validator_execute() - Executing validator SharedStorageNameValidator
2025-12-18 17:14:48,199 - DEBUG - common.py:190:_validator_execute() - Executing validator SharedStorageMountDirValidator
2025-12-18 17:14:48,199 - DEBUG - common.py:190:_validator_execute() - Executing validator FeatureRegionValidator
2025-12-18 17:14:48,199 - DEBUG - common.py:190:_validator_execute() - Executing validator FsxArchitectureOsValidator
2025-12-18 17:14:48,199 - DEBUG - common.py:190:_validator_execute() - Executing validator ExistingFsxNetworkingValidator
2025-12-18 17:14:48,213 - DEBUG - common.py:190:_validator_execute() - Executing validator NumberOfStorageValidator
2025-12-18 17:14:48,213 - DEBUG - common.py:190:_validator_execute() - Executing validator NumberOfStorageValidator
2025-12-18 17:14:48,213 - DEBUG - common.py:190:_validator_execute() - Executing validator NumberOfStorageValidator
2025-12-18 17:14:48,213 - DEBUG - common.py:190:_validator_execute() - Executing validator NumberOfStorageValidator
2025-12-18 17:14:48,213 - DEBUG - common.py:190:_validator_execute() - Executing validator NumberOfStorageValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator NumberOfStorageValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator NumberOfStorageValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator ManagedFsxMultiAzValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator MultiAzEbsVolumeValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator MultiAzRootVolumeValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator DuplicateMountDirValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator OverlappingMountDirValidator
2025-12-18 17:14:48,214 - DEBUG - common.py:190:_validator_execute() - Executing validator HeadNodeLaunchTemplateValidator
2025-12-18 17:14:48,225 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=RunInstances, params={'InstanceType': 'c6i.large', 'MinCount': 1, 'MaxCount': 1, 'ImageId': 'ami-0aeb582419dd31854', 'NetworkInterfaces': [{'DeviceIndex': 0, 'NetworkCardIndex': 0, 'InterfaceType': 'interface', 'Groups': ['sg-0b6398ec0dbba2ef2'], 'SubnetId': 'subnet-02022469c3451395b'}], 'DryRun': True, 'TagSpecifications': [], 'KeyName': 'REDACTED-eu-north-1', 'BlockDeviceMappings': [{'DeviceName': '/dev/xvdba', 'VirtualName': 'ephemeral0'}, {'DeviceName': '/dev/xvdbb', 'VirtualName': 'ephemeral1'}, {'DeviceName': '/dev/xvdbc', 'VirtualName': 'ephemeral2'}, {'DeviceName': '/dev/xvdbd', 'VirtualName': 'ephemeral3'}, {'DeviceName': '/dev/xvdbe', 'VirtualName': 'ephemeral4'}, {'DeviceName': '/dev/xvdbf', 'VirtualName': 'ephemeral5'}, {'DeviceName': '/dev/xvdbg', 'VirtualName': 'ephemeral6'}, {'DeviceName': '/dev/xvdbh', 'VirtualName': 'ephemeral7'}, {'DeviceName': '/dev/xvdbi', 'VirtualName': 'ephemeral8'}, {'DeviceName': '/dev/xvdbj', 'VirtualName': 'ephemeral9'}, {'DeviceName': '/dev/xvdbk', 'VirtualName': 'ephemeral10'}, {'DeviceName': '/dev/xvdbl', 'VirtualName': 'ephemeral11'}, {'DeviceName': '/dev/xvdbm', 'VirtualName': 'ephemeral12'}, {'DeviceName': '/dev/xvdbn', 'VirtualName': 'ephemeral13'}, {'DeviceName': '/dev/xvdbo', 'VirtualName': 'ephemeral14'}, {'DeviceName': '/dev/xvdbp', 'VirtualName': 'ephemeral15'}, {'DeviceName': '/dev/xvdbq', 'VirtualName': 'ephemeral16'}, {'DeviceName': '/dev/xvdbr', 'VirtualName': 'ephemeral17'}, {'DeviceName': '/dev/xvdbs', 'VirtualName': 'ephemeral18'}, {'DeviceName': '/dev/xvdbt', 'VirtualName': 'ephemeral19'}, {'DeviceName': '/dev/xvdbu', 'VirtualName': 'ephemeral20'}, {'DeviceName': '/dev/xvdbv', 'VirtualName': 'ephemeral21'}, {'DeviceName': '/dev/xvdbw', 'VirtualName': 'ephemeral22'}, {'DeviceName': '/dev/xvdbx', 'VirtualName': 'ephemeral23'}, {'DeviceName': '/dev/sda1', 'Ebs': {'Encrypted': True, 'VolumeType': 'gp3', 'Iops': 3000, 'Throughput': 125, 'DeleteOnTermination': True}}], 'MetadataOptions': {'HttpTokens': 'required'}}
2025-12-18 17:14:48,852 - DEBUG - common.py:190:_validator_execute() - Executing validator SchedulerValidator
2025-12-18 17:14:48,852 - DEBUG - common.py:190:_validator_execute() - Executing validator SchedulerOsValidator
2025-12-18 17:14:48,852 - DEBUG - common.py:190:_validator_execute() - Executing validator HeadNodeImdsValidator
2025-12-18 17:14:48,852 - DEBUG - common.py:190:_validator_execute() - Executing validator RootVolumeSizeValidator
2025-12-18 17:14:48,853 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeTypeSizeValidator
2025-12-18 17:14:48,853 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeIopsValidator
2025-12-18 17:14:48,853 - DEBUG - common.py:190:_validator_execute() - Executing validator KeyPairValidator
2025-12-18 17:14:48,865 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=DescribeKeyPairs, params={'KeyNames': ['REDACTED-eu-north-1']}
2025-12-18 17:14:49,066 - DEBUG - common.py:190:_validator_execute() - Executing validator MixedSecurityGroupOverwriteValidator
2025-12-18 17:14:49,066 - DEBUG - common.py:190:_validator_execute() - Executing validator MultiNetworkInterfacesInstancesValidator
2025-12-18 17:14:49,066 - DEBUG - common.py:190:_validator_execute() - Executing validator ComputeResourceLaunchTemplateValidator
2025-12-18 17:14:49,079 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=ec2, operation=RunInstances, params={'InstanceType': 'hpc6a.48xlarge', 'MinCount': 1, 'MaxCount': 1, 'ImageId': 'ami-0aeb582419dd31854', 'Placement': {}, 'NetworkInterfaces': [{'DeviceIndex': 0, 'NetworkCardIndex': 0, 'InterfaceType': 'efa', 'Groups': ['sg-0b6398ec0dbba2ef2'], 'SubnetId': 'subnet-0f51e28e2cd594d96'}], 'DryRun': True, 'TagSpecifications': [], 'BlockDeviceMappings': [{'DeviceName': '/dev/xvdba', 'VirtualName': 'ephemeral0'}, {'DeviceName': '/dev/xvdbb', 'VirtualName': 'ephemeral1'}, {'DeviceName': '/dev/xvdbc', 'VirtualName': 'ephemeral2'}, {'DeviceName': '/dev/xvdbd', 'VirtualName': 'ephemeral3'}, {'DeviceName': '/dev/xvdbe', 'VirtualName': 'ephemeral4'}, {'DeviceName': '/dev/xvdbf', 'VirtualName': 'ephemeral5'}, {'DeviceName': '/dev/xvdbg', 'VirtualName': 'ephemeral6'}, {'DeviceName': '/dev/xvdbh', 'VirtualName': 'ephemeral7'}, {'DeviceName': '/dev/xvdbi', 'VirtualName': 'ephemeral8'}, {'DeviceName': '/dev/xvdbj', 'VirtualName': 'ephemeral9'}, {'DeviceName': '/dev/xvdbk', 'VirtualName': 'ephemeral10'}, {'DeviceName': '/dev/xvdbl', 'VirtualName': 'ephemeral11'}, {'DeviceName': '/dev/xvdbm', 'VirtualName': 'ephemeral12'}, {'DeviceName': '/dev/xvdbn', 'VirtualName': 'ephemeral13'}, {'DeviceName': '/dev/xvdbo', 'VirtualName': 'ephemeral14'}, {'DeviceName': '/dev/xvdbp', 'VirtualName': 'ephemeral15'}, {'DeviceName': '/dev/xvdbq', 'VirtualName': 'ephemeral16'}, {'DeviceName': '/dev/xvdbr', 'VirtualName': 'ephemeral17'}, {'DeviceName': '/dev/xvdbs', 'VirtualName': 'ephemeral18'}, {'DeviceName': '/dev/xvdbt', 'VirtualName': 'ephemeral19'}, {'DeviceName': '/dev/xvdbu', 'VirtualName': 'ephemeral20'}, {'DeviceName': '/dev/xvdbv', 'VirtualName': 'ephemeral21'}, {'DeviceName': '/dev/xvdbw', 'VirtualName': 'ephemeral22'}, {'DeviceName': '/dev/xvdbx', 'VirtualName': 'ephemeral23'}, {'DeviceName': '/dev/sda1', 'Ebs': {'Encrypted': True, 'VolumeType': 'gp3', 'Iops': 3000, 'Throughput': 125, 'DeleteOnTermination': True, 'VolumeSize': 200}}], 'MetadataOptions': {'HttpTokens': 'required'}}
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator RootVolumeSizeValidator
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeTypeSizeValidator
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator EbsVolumeIopsValidator
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceArchitectureCompatibilityValidator
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator EfaOsArchitectureValidator
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator PlacementGroupCapacityTypeValidator
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceTypeBaseAMICompatibleValidator
2025-12-18 17:14:49,664 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceTypeOSCompatibleValidator
2025-12-18 17:14:49,665 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceTypeAcceleratorManufacturerValidator
2025-12-18 17:14:49,665 - DEBUG - common.py:190:_validator_execute() - Executing validator InstanceTypePlacementGroupValidator
2025-12-18 17:14:49,665 - DEBUG - common.py:190:_validator_execute() - Executing validator ComputeResourceTagsValidator
2025-12-18 17:14:49,665 - DEBUG - common.py:190:_validator_execute() - Executing validator HeadNodeMemorySizeValidator
2025-12-18 17:14:49,665 - INFO - cluster.py:499:_validate_and_parse_config() - Validation succeeded.
2025-12-18 17:14:49,685 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=sts, operation=GetCallerIdentity, params={}
2025-12-18 17:14:50,421 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=s3, operation=HeadBucket, params={'Bucket': 'parallelcluster-redacted-v1-do-not-delete'}
2025-12-18 17:14:50,987 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=s3, operation=HeadObject, params={'Bucket': 'parallelcluster-redacted-v1-do-not-delete', 'Key': 'parallelcluster/.bootstrapped'}
2025-12-18 17:14:51,177 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=s3, operation=GetObject, params={'Bucket': 'parallelcluster-redacted-v1-do-not-delete', 'Key': 'parallelcluster/.bootstrapped'}
2025-12-18 17:14:51,368 - INFO - s3_bucket.py:468:_check_default_bucket() - Bucket parallelcluster-redacted-v1-do-not-delete is already bootstrapped with required features.
2025-12-18 17:14:51,375 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=s3, operation=GetObject, params={'Bucket': 'parallelcluster-redacted-v1-do-not-delete', 'Key': 'parallelcluster/3.14.0/clusters/repro-cluster-5e6268jxpmpaw3c6/configs/cluster-config.yaml', 'VersionId': '51KNj75egNsWlYFOsrubTZsDx6hcY4_s'}
2025-12-18 17:14:51,673 - INFO - common.py:134:_log_boto3_calls() - Executing boto3 call: region=eu-north-1, service=dynamodb, operation=GetItem, params={'TableName': 'parallelcluster-repro-cluster', 'ConsistentRead': True, 'Key': {'Id': 'COMPUTE_FLEET'}}
2025-12-18 17:14:52,258 - CRITICAL - cluster.py:968:update() - 'str' object has no attribute 'get'
2025-12-18 17:14:52,258 - ERROR - entrypoint.py:284:main() - {"message": "Cluster update failed.\n'str' object has no attribute 'get'"}
Traceback (most recent call last):
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/models/cluster.py", line 916, in update
target_config, changes, ignored_validation_failures = self.validate_update_request(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
target_source_config, validator_suppressors, validation_failure_level, force
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/models/cluster.py", line 876, in validate_update_request
changes = self._validate_patch(force, target_config)
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/models/cluster.py", line 884, in _validate_patch
patch_allowed, update_changes = patch.check()
~~~~~~~~~~~^^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/config/config_patch.py", line 259, in check
check_result, reason, action_needed, print_change = change.update_policy.check(change, self)
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/config/update_policy.py", line 80, in check
if self.condition_checker(change, patch):
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/config/update_policy.py", line 473, in condition_checker_shared_storage_update_policy
and not is_compute_fleet_update_supported_for_shared_storage(change)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/config/update_policy.py", line 265, in is_compute_fleet_update_supported_for_shared_storage
change_info = SharedStorageChangeInfo(change)
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/config/update_policy_utils.py", line 25, in __init__
self.storage_type = storage_item.get("StorageType")
^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/api/controllers/common.py", line 149, in wrapper
return func(*args, **kwargs)
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/api/controllers/common.py", line 101, in _wrapper_http_success_status_code
return func(*args, **kwargs), status_code
~~~~^^^^^^^^^^^^^^^^^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/api/controllers/cluster_operations_controller.py", line 391, in update_cluster
changes, ignored_validation_failures = cluster.update(
~~~~~~~~~~~~~~^
target_source_config=cluster_config,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
force=force_update,
^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/models/cluster.py", line 969, in update
raise _cluster_error_mapper(e, f"Cluster update failed.\n{e}")
pcluster.models.cluster.ClusterActionError: Cluster update failed.
'str' object has no attribute 'get'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/entrypoint.py", line 210, in _run_operation
return args.func(args)
~~~~~~~~~^^^^^^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/entrypoint.py", line 116, in dispatch
return middleware[operation](dispatch_func, body, kwargs)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/middleware.py", line 65, in wrapper
ret = func(dest_func, _body, kwargs)
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/middleware.py", line 77, in update_cluster
ret = func(**kwargs)
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/model.py", line 211, in call
ret = func(*args, **kwargs)
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/api/controllers/common.py", line 161, in wrapper
raise InternalServiceException(str(e)) from e
pcluster.api.errors.InternalServiceException

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/entrypoint.py", line 268, in main
ret = run(sys.argv[1:])
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/entrypoint.py", line 262, in run
return _run_operation(model, args, extra_args)
File "/Users/REDACTED/workspace/pcluster/.pcluster3140/lib/python3.13/site-packages/pcluster/cli/entrypoint.py", line 221, in _run_operation
raise APIOperationException(json.loads(error_encoded))
pcluster.cli.exceptions.APIOperationException: {
"message": "Cluster update failed.\n'str' object has no attribute 'get'"
}

```

### Explanation

`SharedStorageChangeInfo` is designed to handle changes at the **SharedStorage item level** (adding/removing entire storage items), where `change.old_value` and `change.new_value` are dictionaries.

However, `SHARED_STORAGE_UPDATE_POLICY` is also applied to **nested property changes** like `MountDir`, where values are strings:

```python
# Expected by SharedStorageChangeInfo: SharedStorage item change
change.key = "SharedStorage"
change.old_value = {"Name": "fsx", "MountDir": "/fsx", "StorageType": "FsxLustre", ...}

# Actual: MountDir property change
change.key = "MountDir"
change.old_value = "/fsx" # string
change.new_value = "/fsx1" # string
```

The code calls `.get("StorageType")` on a string, causing the `AttributeError`.

### Suggested Fix

Add type checking in `SharedStorageChangeInfo.__init__()` or in the calling functions:

**Option 1: Guard in SharedStorageChangeInfo**
```python
def __init__(self, change):
old_value = change.old_value
new_value = change.new_value

storage_item = new_value if new_value is not None else old_value

# Guard against non-dict values (nested property changes)
if not isinstance(storage_item, dict):
self.storage_type = None
self.storage_settings = {}
self.is_external = False
self.is_mount = False
self.is_unmount = False
return

# ... rest of existing code
```

**Option 2: Guard in calling functions**
```python
def is_compute_fleet_update_supported_for_shared_storage(change):
# Only process SharedStorage item-level changes
if not isinstance(change.old_value, dict) and not isinstance(change.new_value, dict):
return False
change_info = SharedStorageChangeInfo(change)
return change_info.is_external and (...)
```

## Additional context

- The same issue likely affects `is_login_fleet_update_supported_for_shared_storage()` at line 251
- Other fields using `SHARED_STORAGE_UPDATE_POLICY` may also be affected (check `cluster_schema.py`)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.