[HiTL] Spin up more runs with different random seed in nsp model retrain infra

## Type of Issue

Select the type of issue:
- [ ] Bug report (to report a bug)
- [x] Feature request (to request an additional feature)
- [ ] Tracker (I am just using this as a tracker)
- [ ] Refactor request
- [ ] Documentation Ask

## Description

Currently we spin up 6 runs with different parameters in model sweep. Those are with same random seed (batch_id).

As discussed, we should expand sweep size -- use N different random seeds, each with 6 runs (the parameter config file should be the same for those N random seeds). The total number of runs in one sweep would be N * 6.

Note that we should use the same mask splits for all the runs in one sweep. That is, keep using batch_id as random seed while generating mask splits (or load & store), then change random seeds to spin up sweep runs.

## Checklist

Use this checklist if you are using this issue as a tracker:

- [ ] Task 1
- [ ] Task 2
- [ ] Task 3 ...