PyPSA / PyPSA/pypsa-eur

cplex issue sector_model pypsa 1.0.6

Open
#1,964 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
615
Forks
459
Avg merge
4d 14h
Merged PRs (30d)
6

Description

Version Checks (indicate both or one)
  • I have confirmed this bug exists on the lastest release of PyPSA-Eur.

  • I have confirmed this bug exists on the current master branch of PyPSA-Eur.

Issue Description

Dear pypsa team,
I am running a sector model elec+hydrogen full eu with 50 nodes only. The network was created using:
snakemake -call solve_sector_networks --configfile config/config.yaml

Then, I am using cplex in a simple python script
import pypsa
file="base_s_50__2190SEG_2040.nc"
n = pypsa.Network(file)
n.snapshots = n.snapshots[1:10]
n.optimize(solver_name="cplex")

leading to
CPLEX Error 1615: Line 388524: Expected number, found 'i'.
.....
cplex.exceptions.errors.CplexSolverError: CPLEX Error 1615: Line 388524: Expected number, found 'i'.

The following can bypass the problem but leading to the loss of storage constraints
import pypsa
file="base_s_50__2190SEG_2040.nc"
n = pypsa.Network(file)
n.snapshots = n.snapshots[1:10]
m=n.optimize.create_model()
m.remove_constraints("Generator-e_sum_min")
m.remove_constraints("Generator-e_sum_max")
m.remove_constraints("Store-fix-e-lower")
m.remove_constraints("Store-fix-e-upper")
m.solve(solver_name="cplex")

leading to
Version identifier: 22.1.2.0 | 2024-12-10 | f4cec290b
CPXPARAM_Read_DataCheck 1
Parallel mode: deterministic, using up to 32 threads for concurrent optimization:

  • Starting dual Simplex on 1 thread...
  • Starting Barrier on 30 threads...
  • Starting primal Simplex on 1 thread...
    Tried aggregator 1 time.
    LP Presolve eliminated 68032 rows and 7772 columns.
    Aggregator did 6013 substitutions.
    Reduced LP has 37258 rows, 41603 columns, and 146107 nonzeros.
    Presolve time = 0.15 sec. (97.25 ticks)
    Initializing dual steep norms . . .

Iteration log . . .
Iteration: 1 Dual objective = -41.087578
Perturbation started.
Iteration: 202 Dual objective = 15590791.115753
Iteration: 674 Dual objective = 1821272328.160855
Iteration: 1022 Dual objective = 3428977391.523224
Iteration: 1538 Dual objective = 8932802333.381388
Iteration: 2000 Dual objective = 12838026214.998688
Iteration: 2483 Dual objective = 16386479955.779938
Iteration: 2923 Dual objective = 18929981925.852730
Iteration: 3427 Dual objective = 21152120362.608200
Iteration: 3866 Dual objective = 24672867061.932739
Iteration: 4233 Dual objective = 25971083657.120106
Iteration: 4582 Dual objective = 27165015506.286640
Iteration: 5035 Dual objective = 29007971842.636559
Iteration: 5345 Dual objective = 29015576437.803383
Iteration: 5685 Dual objective = 30851651844.028008
Iteration: 5987 Dual objective = 31749907160.570126
Iteration: 6311 Dual objective = 31883655943.376015
Iteration: 6584 Dual objective = 31963435089.963665
Iteration: 6962 Dual objective = 33180581760.799374
Iteration: 7454 Dual objective = 35072659236.459312
Iteration: 7801 Dual objective = 35564425527.435173
Iteration: 8053 Dual objective = 35564425536.948059
Iteration: 8275 Dual objective = 35564431153.769653
Iteration: 8513 Dual objective = 35569006748.774162
Iteration: 8930 Dual objective = 36317000946.014244
Iteration: 9401 Dual objective = 37855824450.449181
Iteration: 9641 Dual objective = 37930270836.916565
Iteration: 9871 Dual objective = 37930270851.993240
Iteration: 10086 Dual objective = 37930270869.165543
Iteration: 10321 Dual objective = 37930270888.276146
Removing perturbation.

Barrier solved model.

INFO:linopy.constants: Optimization successful:
Status: ok
Termination condition: optimal
Solution: 55388 primals, 111303 duals
Objective: 8.78e+10
Solver model: available
Solver message: optimal

Reproducible Example
import pypsa
file="base_s_50__2190SEG_2040.nc"
n = pypsa.Network(file)
n.snapshots = n.snapshots[1:10]
n.optimize(solver_name="cplex")

leading to 
CPLEX Error  1615: Line 388524: Expected number, found 'i'.
.....
cplex.exceptions.errors.CplexSolverError: CPLEX Error  1615: Line 388524: Expected number, found 'i'.
Expected Behavior

cplex should run smoothly without having to remove important constraints

Installed Versions
channels: - conda-forge - ibmdecisionoptimization - pkgs/main dependencies: - _libgcc_mutex=0.1=conda_forge - _openmp_mutex=4.5=2_gnu - _python_abi3_support=1.0=hd8ed1ab_2 - affine=2.4.0=pyhd8ed1ab_1 - aiohappyeyeballs=2.6.1=pyhd8ed1ab_0 - aiohttp=3.13.3=py312h5d8c7f2_0 - aiosignal=1.4.0=pyhd8ed1ab_0 - atlite=0.4.1=pyhd8ed1ab_1 - attr=2.5.2=h39aace5_0 - attrs=25.4.0=pyhcf101f3_1 - aws-c-auth=0.9.3=hef928c7_0 - aws-c-cal=0.9.13=h2c9d079_1 - aws-c-common=0.12.6=hb03c661_0 - aws-c-compression=0.3.1=h8b1a151_9 - aws-c-event-stream=0.5.7=h28f887f_1 - aws-c-http=0.10.7=ha8fc4e3_5 - aws-c-io=0.23.3=hdaf4b65_5 - aws-c-mqtt=0.13.3=hc63082f_11 - aws-c-s3=0.11.3=h06ab39a_1 - aws-c-sdkutils=0.2.4=h8b1a151_4 - aws-checksums=0.2.7=h8b1a151_5 - aws-crt-cpp=0.35.4=h8824e59_0 - aws-sdk-cpp=1.11.606=h20b40b1_10 - azure-core-cpp=1.16.1=h3a458e0_0 - azure-identity-cpp=1.13.2=h3a5f585_1 - azure-storage-blobs-cpp=12.15.0=h2a74896_1 - azure-storage-common-cpp=12.11.0=h3d7a050_1 - azure-storage-files-datalake-cpp=12.13.0=hf38f1be_1 - backports.zstd=1.3.0=py312h90b7ffd_0 - beautifulsoup4=4.14.3=pyha770c72_0 - blosc=1.21.6=he440d0b_1 - bokeh=3.8.1=pyhd8ed1ab_0 - bottleneck=1.6.0=np2py312hfb8c2c5_3 - branca=0.8.2=pyhd8ed1ab_0 - brotli=1.2.0=hed03a55_1 - brotli-bin=1.2.0=hb03c661_1 - brotli-python=1.2.0=py312hdb49522_1 - bzip2=1.0.8=hda65f42_8 - c-ares=1.34.6=hb03c661_0 - c-blosc2=2.22.0=hc31b594_1 - ca-certificates=2026.1.4=hbd8a1cb_0 - cached-property=1.5.2=hd8ed1ab_1 - cached_property=1.5.2=pyha770c72_1 - cdsapi=0.7.7=pyhd8ed1ab_0 - certifi=2026.1.4=pyhd8ed1ab_0 - cffi=2.0.0=py312h460c074_1 - cfgrib=0.9.15.1=pyhd8ed1ab_0 - cftime=1.6.5=py312h4f23490_0 - charset-normalizer=3.4.4=pyhd8ed1ab_0 - click=8.3.1=pyh8f84b5b_1 - click-plugins=1.1.1.2=pyhd8ed1ab_0 - cligj=0.7.2=pyhd8ed1ab_2 - cloudpickle=3.1.2=pyhcf101f3_1 - colorama=0.4.6=pyhd8ed1ab_1 - contourpy=1.3.3=py312hd9148b4_3 - country_converter=1.3.2=pyhd8ed1ab_0 - cplex=22.1.2=py312_0 - cpython=3.12.12=py312hd8ed1ab_1 - cryptography=46.0.3=py312ha4b625e_1 - cycler=0.12.1=pyhcf101f3_2 - cytoolz=1.1.0=py312h4c3975b_1 - dask=2025.12.0=pyhcf101f3_0 - dask-core=2025.12.0=pyhcf101f3_1 - deprecation=2.1.0=pyh9f0ad1d_0 - distributed=2025.12.0=pyhcf101f3_1 - eccodes=2.44.0=h83bc92c_0 - ecmwf-datastores-client=0.4.1=pyhd8ed1ab_0 - entsoe-py=0.7.8=pyhd8ed1ab_0 - et_xmlfile=2.0.0=pyhd8ed1ab_1 - exceptiongroup=1.3.1=pyhd8ed1ab_0 - findlibs=0.1.2=pyhd8ed1ab_0 - fiona=1.10.1=py312h053e1f3_6 - folium=0.20.0=pyhd8ed1ab_0 - fonttools=4.61.1=py312h8a5da7c_0 - freeglut=3.2.2=ha6d2627_3 - freetype=2.14.1=ha770c72_0 - freexl=2.0.0=h9dce30a_2 - frozenlist=1.7.0=py312h447239a_0 - fsspec=2025.12.0=pyhd8ed1ab_0 - geographiclib=2.1=pyhd8ed1ab_0 - geopandas=1.1.2=pyhd8ed1ab_0 - geopandas-base=1.1.2=pyha770c72_0 - geopy=2.4.1=pyhd8ed1ab_2 - geos=3.14.1=h480dda7_0 - gflags=2.2.2=h5888daf_1005 - giflib=5.2.2=hd590300_0 - glog=0.7.1=hbabe93e_0 - google-api-core=2.28.1=pyhd8ed1ab_0 - google-auth=2.47.0=pyhcf101f3_0 - google-cloud-core=2.5.0=py312h06a4308_0 - google-cloud-storage=3.7.0=pyhcf101f3_0 - google-crc32c=1.8.0=py312h03f33d3_0 - google-resumable-media=2.8.0=pyhd8ed1ab_0 - googleapis-common-protos=1.72.0=pyhd8ed1ab_0 - h2=4.3.0=pyhcf101f3_0 - h5netcdf=1.7.3=pyhd8ed1ab_0 - h5py=3.15.1=nompi_py312ha4f8f14_101 - hdf4=4.2.15=h2a13503_7 - hdf5=1.14.6=nompi_h1b119a7_104 - highspy=1.12.0=np2py312h0f77346_0 - hpack=4.1.0=pyhd8ed1ab_0 - hyperframe=6.1.0=pyhd8ed1ab_0 - icu=78.1=h33c6efd_0 - idna=3.11=pyhd8ed1ab_0 - importlib-metadata=8.7.0=pyhe01879c_1 - iniconfig=2.3.0=pyhd8ed1ab_0 - jasper=4.2.8=he3c4edf_0 - jinja2=3.1.6=pyhcf101f3_1 - joblib=1.5.3=pyhd8ed1ab_0 - json-c=0.18=h6688a6e_0 - keyutils=1.6.3=hb9d3cd8_0 - kiwisolver=1.4.9=py312h0a2e395_2 - krb5=1.21.3=h659f571_0 - lcms2=2.17=h717163a_0 - ld_impl_linux-64=2.45=default_hbd61a6d_105 - legacy-cgi=2.6.4=pyhcf101f3_0 - lerc=4.0.0=h0aef613_1 - levenshtein=0.27.3=py312h1289d80_0 - libabseil=20250512.1=cxx17_hba17884_0 - libaec=1.1.4=h3f801dc_0 - libarchive=3.8.5=gpl_hc2c16d8_100 - libarrow=22.0.0=hb6ed5f4_6_cpu - libarrow-acero=22.0.0=h635bf11_6_cpu - libarrow-compute=22.0.0=h8c2c5c3_6_cpu - libarrow-dataset=22.0.0=h635bf11_6_cpu - libarrow-substrait=22.0.0=h3f74fd7_6_cpu - libblas=3.11.0=5_h4a7cf45_openblas - libbrotlicommon=1.2.0=hb03c661_1 - libbrotlidec=1.2.0=hb03c661_1 - libbrotlienc=1.2.0=hb03c661_1 - libcblas=3.11.0=5_h0358290_openblas - libcrc32c=1.1.2=h9c3ff4c_0 - libcurl=8.17.0=h4e3cde8_1 - libdeflate=1.25=h17f619e_0 - libedit=3.1.20250104=pl5321h7949ede_0 - libev=4.33=hd590300_2 - libevent=2.1.12=hf998b51_1 - libexpat=2.7.3=hecca717_0 - libffi=3.5.2=h9ec8514_0 - libfreetype=2.14.1=ha770c72_0 - libfreetype6=2.14.1=h73754d4_0 - libgcc=15.2.0=he0feb66_16 - libgcc-ng=15.2.0=h69a702a_16 - libgdal-core=3.12.1=hf05ffb4_0 - libgfortran=15.2.0=h69a702a_16 - libgfortran5=15.2.0=h68bc16d_16 - libglu=9.0.3=h5888daf_1 - libglvnd=1.7.0=ha4b6fd6_2 - libgomp=15.2.0=he0feb66_16 - libgoogle-cloud=2.39.0=hdb79228_0 - libgoogle-cloud-storage=2.39.0=hdbdcf42_0 - libgrpc=1.73.1=h3288cfb_1 - libhwy=1.3.0=h4c17acf_1 - libiconv=1.18=h3b78370_2 - libjpeg-turbo=3.1.2=hb03c661_0 - libjxl=0.11.1=hf08fa70_5 - libkml=1.3.0=haa4a5bd_1022 - liblapack=3.11.0=5_h47877c9_openblas - liblzma=5.8.1=hb9d3cd8_2 - libnetcdf=4.9.3=nompi_h11f7409_103 - libnghttp2=1.67.0=had1ee68_0 - libnsl=2.0.1=hb9d3cd8_1 - libopenblas=0.3.30=pthreads_h94d23a6_4 - libopengl=1.7.0=ha4b6fd6_2 - libopentelemetry-cpp=1.21.0=hb9b0907_1 - libopentelemetry-cpp-headers=1.21.0=ha770c72_1 - libparquet=22.0.0=h7376487_6_cpu - libpng=1.6.53=h421ea60_0 - libprotobuf=6.31.1=h49aed37_4 - libre2-11=2025.11.05=h7b12aa8_0 - librttopo=1.1.0=h46dd2a8_20 - libspatialite=5.1.0=gpl_h2abfd87_119 - libsqlite=3.51.1=hf4e2dac_1 - libssh2=1.11.1=hcf80075_0 - libstdcxx=15.2.0=h934c35e_16 - libstdcxx-ng=15.2.0=hdf11a46_16 - libthrift=0.22.0=h454ac66_1 - libtiff=4.7.1=h9d88235_1 - libutf8proc=2.11.2=hfe17d71_0 - libuuid=2.41.3=h5347b49_0 - libwebp-base=1.6.0=hd42ef1d_0 - libxcb=1.17.0=h8a09558_0 - libxcrypt=4.4.36=hd590300_1 - libxml2=2.15.1=he237659_1 - libxml2-16=2.15.1=hca6bf5a_1 - libxml2-devel=2.15.1=he237659_1 - libzip=1.11.2=h6991a6a_0 - libzlib=1.3.1=hb9d3cd8_2 - linopy=0.5.7=pyhd8ed1ab_0 - locket=1.0.0=pyhd8ed1ab_0 - lz4=4.4.5=py312h3d67a73_1 - lz4-c=1.10.0=h5888daf_1 - lzo=2.10=h280c20c_1002 - mapclassify=2.10.0=pyhd8ed1ab_1 - markupsafe=3.0.3=py312h8a5da7c_0 - matplotlib-base=3.10.8=py312he3d6523_0 - memory_profiler=0.61.0=pyhcf101f3_1 - minizip=4.0.10=h05a5f5f_0 - msgpack-python=1.1.2=py312hd9148b4_1 - multidict=6.7.0=py312h8a5da7c_0 - multiurl=0.3.7=pyhd8ed1ab_0 - munkres=1.1.4=pyhd8ed1ab_1 - muparser=2.3.5=h5888daf_0 - narwhals=2.15.0=pyhcf101f3_0 - ncurses=6.5=h2d0b736_3 - netcdf4=1.7.3=nompi_py312hf6400b3_100 - networkx=3.6.1=pyhcf101f3_0 - nlohmann_json=3.12.0=h54a6638_1 - nomkl=1.0=h5ca1d4c_0 - numexpr=2.14.1=py312h88efc94_101 - numpy=1.26.4=py312heda63a1_0 - openjpeg=2.5.4=h55fea9a_0 - openpyxl=3.1.5=py312h7f6eeab_2 - openssl=3.6.0=h26f9b46_0 - orc=2.2.1=hd747db4_0 - packaging=25.0=pyh29332c3_1 - pandas=2.3.3=py312hf79963d_1 - partd=1.4.2=pyhd8ed1ab_0 - patsy=1.0.2=pyhcf101f3_0 - pcre2=10.47=haa7fec5_0 - pillow=12.1.0=py312h50c33e8_0 - pip=25.3=pyh8b19718_0 - plotly=6.5.0=pyhd8ed1ab_0 - pluggy=1.6.0=pyhf9edf01_1 - polars=1.36.1=pyh6a1acc5_0 - polars-runtime-32=1.36.1=py310hffdcd12_0 - powerplantmatching=0.7.1=pyhd8ed1ab_0 - progressbar2=4.5.0=pyhd8ed1ab_1 - proj=9.7.1=h99ae125_0 - prometheus-cpp=1.3.0=ha5d0236_0 - propcache=0.3.1=py312h178313f_0 - proto-plus=1.27.0=pyhd8ed1ab_0 - protobuf=6.31.1=py312hb8af0ac_2 - psutil=7.2.1=py312h5253ce2_0 - pthread-stubs=0.4=hb9d3cd8_1002 - py-cpuinfo=9.0.0=pyhd8ed1ab_1 - pyarrow=22.0.0=py312h7900ff3_0 - pyarrow-core=22.0.0=py312hc195796_0_cpu - pyasn1=0.6.1=pyhd8ed1ab_2 - pyasn1-modules=0.4.2=pyhd8ed1ab_0 - pycountry=24.6.1=pyhd8ed1ab_0 - pycparser=2.22=pyh29332c3_1 - pydeck=0.9.1=pyhd8ed1ab_0 - pygments=2.19.2=pyhd8ed1ab_0 - pyogrio=0.12.1=py312h053e1f3_0 - pyopenssl=25.3.0=pyhd8ed1ab_0 - pyparsing=3.3.1=pyhcf101f3_0 - pyproj=3.7.2=py312h9b6a7d9_2 - pypsa=1.0.6=pyhd8ed1ab_1 - pysocks=1.7.1=pyha55dd90_7 - pytables=3.10.2=py312hefc0c3f_10 - pytest=9.0.2=pyhcf101f3_0 - python=3.12.12=hd63d673_1_cpython - python-dateutil=2.9.0.post0=pyhe01879c_2 - python-eccodes=2.44.0=py312h4f23490_1 - python-gil=3.12.12=hd8ed1ab_1 - python-tzdata=2025.3=pyhd8ed1ab_0 - python-utils=3.9.1=pyhff2d567_1 - python_abi=3.12=8_cp312 - pytz=2025.2=pyhd8ed1ab_0 - pyu2f=0.1.5=pyhd8ed1ab_1 - pyyaml=6.0.3=py312h8a5da7c_0 - qhull=2020.2=h434a139_5 - rapidfuzz=3.14.3=py312h1289d80_1 - rasterio=1.5.0=py312hcedc861_0 - re2=2025.11.05=h5301d42_0 - readline=8.3=h853b02a_0 - requests=2.32.5=pyhcf101f3_1 - rsa=4.9.1=pyhd8ed1ab_0 - s2n=1.6.2=he8a4886_1 - scikit-learn=1.8.0=np2py312h3226591_1 - scipy=1.16.3=py312h54fa4ab_2 - seaborn=0.13.2=hd8ed1ab_3 - seaborn-base=0.13.2=pyhd8ed1ab_3 - setuptools=80.9.0=pyhff2d567_0 - shapely=2.1.2=py312h383787d_2 - six=1.17.0=pyhe01879c_1 - snappy=1.2.2=h03e3b7b_1 - snuggs=1.4.7=pyhd8ed1ab_2 - sortedcontainers=2.4.0=pyhd8ed1ab_1 - soupsieve=2.8.1=pyhd8ed1ab_0 - sqlite=3.51.1=h04a0ce9_1 - statsmodels=0.14.6=py312h4f23490_0 - tblib=3.2.2=pyhcf101f3_0 - threadpoolctl=3.6.0=pyhecae5ae_0 - tk=8.6.13=noxft_ha0e22de_103 - tomli=2.3.0=pyhcf101f3_0 - toolz=1.1.0=pyhd8ed1ab_1 - tornado=6.5.3=py312h4c3975b_0 - tqdm=4.67.1=pyhd8ed1ab_1 - typing-extensions=4.15.0=h396c80c_0 - typing_extensions=4.15.0=pyhcf101f3_0 - tzdata=2025c=hc9c84f9_1 - unicodedata2=17.0.0=py312h4c3975b_1 - unidecode=1.4.0=pyhcf101f3_1 - uriparser=0.9.8=hac33072_0 - urllib3=2.6.2=pyhd8ed1ab_0 - validators=0.35.0=pyhd8ed1ab_0 - wheel=0.45.1=pyhd8ed1ab_1 - xarray=2025.12.0=pyhcf101f3_0 - xerces-c=3.3.0=hd9031aa_1 - xlrd=2.0.2=pyhd8ed1ab_0 - xorg-libx11=1.8.12=h4f16b4b_0 - xorg-libxau=1.0.12=hb03c661_1 - xorg-libxdmcp=1.1.5=hb03c661_1 - xorg-libxext=1.3.6=hb9d3cd8_0 - xorg-libxfixes=6.0.2=hb03c661_0 - xorg-libxi=1.8.2=hb9d3cd8_0 - xyzservices=2025.11.0=pyhd8ed1ab_0 - yaml=0.2.5=h280c20c_3 - yarl=1.22.0=py312h8a5da7c_0 - zict=3.0.0=pyhd8ed1ab_1 - zipp=3.23.0=pyhcf101f3_1 - zlib=1.3.1=hb9d3cd8_2 - zlib-ng=2.3.2=hceb46e0_1 - zstd=1.5.7=hb78ec9c_6 - pip: - Cartopy==0.24.1 - basemap==1.4.1 - geopandas==1.0.1 - geos==0.2.3 - powerplantmatching==0.6.0 - pypsa==0.31.0 - snakemake==9.14.5 - snakemake-executor-plugin-cluster-generic==1.0.9 - snakemake-storage-plugin-cached-http==0.1.0 - snakemake-storage-plugin-http==0.3.0

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with config/config.yaml, solve_sector_networks, and the provided Python script calling n.optimize(solver_name="cplex"). Inspect the optimization path around the listed storage constraints and the CPLEX failure. Done means the sector model solves with CPLEX without removing storage constraints.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.