AI4OPT / AI4OPT/PGLearn.jl

How to split a sampling task into multiple job submissions

Open
#210 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
7
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Using slurm/submit_jobs.jl, one can generate the sbatch scripts for sampling `n_samples` problem instances and solving the specified problem formulations for these instances, evenly divided across `n_jobs` jobs, as specified in the configuration TOML file.

There may be limits on the running time of each job or the number of total jobs one can submit at once. This makes it sometimes unreliable to run jobs involving solving the `SparseSDPOPF` formulation since it takes more significant amount of time.

Therefore, I wonder if there is any way to split a sampling task into multiple job submissions, each time only running a fraction of the total `n_samples` samples, and merge them in the end.

My current workaround is
1. setting `n_jobs` to be greater than the maximum number of jobs I can submit at once
2. modify sampler.sbatch manually to run only a fraction of the total`n_jobs` jobs (by modifying `--array`)
3. manually submit sampler.sbatch
4. wait for the jobs to finish, and repeat steps 2 & 3
5. finally manually submit extract.sbatch

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.