Incorrect memory allocation for `add_demand`

Feature Request

We need to add in a conditional for memory allocation in add_demand to check if its a sector study. By using all input files, the memory requirements for sector studies (which only reads in one demand at a time) is way high! 1.5TB in this case, but we only use 84 GB.

[Mon Feb 17 18:56:04 2025]
Finished job 121.
1 of 21 steps (5%) done
Select jobs to execute...

[Mon Feb 17 18:56:05 2025]
rule add_demand:
    input: resources/USA/usa/elec_base_network.nc, resources/USA/usa/demand/residential_electricity.pkl, resources/USA/usa/demand/residential_cooling.pkl, resources/USA/usa/demand/residential_space-heating.pkl, resources/USA/usa/demand/residential_water-heating.pkl, resources/USA/usa/demand/commercial_electricity.pkl, resources/USA/usa/demand/commercial_cooling.pkl, resources/USA/usa/demand/commercial_space-heating.pkl, resources/USA/usa/demand/commercial_water-heating.pkl, resources/USA/usa/demand/industry_electricity.pkl, resources/USA/usa/demand/industry_heating.pkl, resources/USA/usa/demand/transport_light-duty_lpg.pkl, resources/USA/usa/demand/transport_light-duty_electricity.pkl, resources/USA/usa/demand/transport_med-duty_lpg.pkl, resources/USA/usa/demand/transport_med-duty_electricity.pkl, resources/USA/usa/demand/transport_heavy-duty_lpg.pkl, resources/USA/usa/demand/transport_heavy-duty_electricity.pkl, resources/USA/usa/demand/transport_bus_lpg.pkl, resources/USA/usa/demand/transport_bus_electricity.pkl, resources/USA/usa/demand/transport_boat-shipping_lpg.pkl, resources/USA/usa/demand/transport_rail-shipping_lpg.pkl, resources/USA/usa/demand/transport_rail-passenger_lpg.pkl, resources/USA/usa/demand/transport_air_lpg.pkl
    output: resources/USA/usa/elec_base_network_dem.nc
    log: logs/USA/usa/add_demand.log
    jobid: 20
    benchmark: benchmarks/USA/usa/add_demand
    reason: Missing output files: resources/USA/usa/elec_base_network_dem.nc; Input files updated by another job: resources/USA/usa/demand/transport_med-duty_electricity.pkl, resources/USA/usa/demand/transport_med-duty_lpg.pkl, resources/USA/usa/demand/transport_light-duty_lpg.pkl, resources/USA/usa/demand/transport_bus_lpg.pkl, resources/USA/usa/demand/transport_light-duty_electricity.pkl, resources/USA/usa/demand/transport_heavy-duty_electricity.pkl, resources/USA/usa/demand/transport_heavy-duty_lpg.pkl, resources/USA/usa/demand/transport_bus_electricity.pkl
    wildcards: interconnect=usa
    resources: mem_mb=1590618, mem_mib=1516932, disk_mb=106185, disk_mib=101266, tmpdir=<TBD>

Submitted job 20 with external jobid 'Submitted batch job 54867276'

$ seff 54867276
Job ID: 54867276
Cluster: cedar
User/Group: 
State: TIMEOUT (exit code 0)
Cores: 1
CPU Utilized: 03:51:58
CPU Efficiency: 96.61% of 04:00:06 core-walltime
Job Wall-clock time: 04:00:06
Memory Utilized: 84.31 GB
Memory Efficiency: 5.43% of 1.52 TB (1.52 TB/node)

Suggested Solution

No response

Additional Info

No response

Locate the add_demand entry point and trace how memory is calculated for sector studies versus other studies. Reproduce the sector-study case using the issue's resource and seff output, then verify that the requested allocation reflects the one-demand-at-a-time behavior without changing other cases.