iiasa / iiasa/message_ix

Migrating away from GAMS

Open
#879 6 comments 2 reactions 0 assignees View on GitHub
enh gams
Dominant language
Jupyter Notebook
Stars
150
Forks
178
Avg merge
17h 32m
Merged PRs (30d)
2

Description

### What is this about?

Since GAMS is proprietary software, receiving one of the expensive licenses is often a bottleneck for using message_ix, especially for researchers from the Global South. It would thus greatly benefit our inclusivity and bolster our open source spirit if we moved away from GAMS.

There are several replacement options: [pyomo](https://github.com/Pyomo/pyomo) and [linopy](https://github.com/PyPSA/linopy) come to mind most readily. I saw a presentation about linopy and a performance comparison during the OpenMod workshop hosted at IIASA last year, which is why I chose linopy (using the [HiGHS](https://github.com/ERGO-Code/HiGHS) solver underneath) to implement a proof of concept for the [ixmp4 transport tutorial](https://github.com/iiasa/ixmp4/pull/101). A similar migration could be done for the actual MESSAGE GAMS code, though this comprises a lot of code and would thus likely require a significant time investment.

During today's weekly meeting, I brought up my wish to eventually perform this migration. Several colleagues shared their experiences and made suggestions right away:

- @behnam-zakeri shared that we could look into [gamspy](https://www.gams.com/blog/2023/11/introducing-gamspy/), too. This, together with GAMS' move to [make licenses free for academic institutions](https://forum.gams.com/t/exciting-news-for-the-academic-community-free-gamspy-available-now/7183) would present an alternative to using linopy/HiGHS. We would still have a python interface to the GAMS code, while possibly avoiding the actual rewriting of our GAMS code and likely avoiding the workflow problem presented by @SiddharthJoshi-Git below. The downside would be that our results would still not be completely publicly reproducible and we would still support the closed-source GAMS.
- @SiddharthJoshi-Git shared part of his usual workflow: for quick assessment of results or copying of data, he often uses the `.gdx` files, which seem to be much more performant than dumping the results to Excel or `.csv`. Our migration should take this into account and provide ways to quickly and conveniently access the same information stored in the `.gdx` as well as the `.log` and `.lst` files.
- @yiyi1991 expressed her support for this migration and stated that the [OseMOSYS team has already done something similar](https://github.com/OSeMOSYS/OSeMOSYS_linopy). This might be a place to look for examples as well as an opportunity for collaboration.
- @khaeru noted that we would need to ensure before migration that all GAMS features we use are also available with our choice of tools and that performance is at least at comparable levels.

All of this means that we're not likely to start this migration anytime soon, much less finish it, but I wanted to record this discussion. For future reference and for you, dear enthusiastic community member: please reach out to us to get information on the current status of this project! Any help will be greatly appreciated :heart:

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the ixmp4 transport tutorial proof of concept in PR 101 and the OSeMOSYS_linopy project mentioned in the issue. Inventory the GAMS features and the .gdx, .log, and .lst access patterns that must be retained, then compare the proposed replacement options. Done would require an agreed migration plan with comparable functionality and performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.