how set_transform affects batch size?

### Describe the bug

I am trying to fine-tune w2v-bert for ASR task. Since my dataset is so big, I preferred to use the on-the-fly method with set_transform. So i change the preprocessing function to this:
```
def prepare_dataset(batch):
input_features = processor(batch["audio"], sampling_rate=16000).input_features[0]
input_length = len(input_features)
labels = processor.tokenizer(batch["text"], padding=False).input_ids

batch = {
"input_features": [input_features],
"input_length": [input_length],
"labels": [labels]
}

return batch

train_ds.set_transform(prepare_dataset)
val_ds.set_transform(prepare_dataset)
```

After this, I also had to change the DataCollatorCTCWithPadding class like this:
```
@dataclass
class DataCollatorCTCWithPadding:
processor: Wav2Vec2BertProcessor
padding: Union[bool, str] = True

def __call__(self, features: List[Dict[str, Union[List[int], torch.Tensor]]]) -> Dict[str, torch.Tensor]:
# Separate input_features and labels
input_features = [{"input_features": feature["input_features"][0]} for feature in features]
labels = [feature["labels"][0] for feature in features]

# Pad input features
batch = self.processor.pad(
input_features,
padding=self.padding,
return_tensors="pt",
)

# Pad and process labels
label_features = self.processor.tokenizer.pad(
{"input_ids": labels},
padding=self.padding,
return_tensors="pt",
)

labels = label_features["input_ids"]
attention_mask = label_features["attention_mask"]

# Replace padding with -100 to ignore these tokens during loss calculation
labels = labels.masked_fill(attention_mask.ne(1), -100)

batch["labels"] = labels

return batch
```

But now a strange thing is happening, no matter how much I increase the batch size, the amount of V-RAM GPU usage does not change, while the number of total steps in the progress-bar (logging) changes. Is this normal or have I made a mistake?

### Steps to reproduce the bug

i can share my code if needed

### Expected behavior

Equal to the batch size value, the set_transform function is applied to the dataset and given to the model as a batch.

### Environment info

all updated versions