Migrating away from GAMS
- Dominant language
- Jupyter Notebook
- Stars
- 150
- Forks
- 178
- Avg merge
- 17h 32m
- Merged PRs (30d)
- 2
Description
### What is this about?
Since GAMS is proprietary software, receiving one of the expensive licenses is often a bottleneck for using message_ix, especially for researchers from the Global South. It would thus greatly benefit our inclusivity and bolster our open source spirit if we moved away from GAMS.
There are several replacement options: [pyomo](https://github.com/Pyomo/pyomo) and [linopy](https://github.com/PyPSA/linopy) come to mind most readily. I saw a presentation about linopy and a performance comparison during the OpenMod workshop hosted at IIASA last year, which is why I chose linopy (using the [HiGHS](https://github.com/ERGO-Code/HiGHS) solver underneath) to implement a proof of concept for the [ixmp4 transport tutorial](https://github.com/iiasa/ixmp4/pull/101). A similar migration could be done for the actual MESSAGE GAMS code, though this comprises a lot of code and would thus likely require a significant time investment.
During today's weekly meeting, I brought up my wish to eventually perform this migration. Several colleagues shared their experiences and made suggestions right away:
- @behnam-zakeri shared that we could look into [gamspy](https://www.gams.com/blog/2023/11/introducing-gamspy/), too. This, together with GAMS' move to [make licenses free for academic institutions](https://forum.gams.com/t/exciting-news-for-the-academic-community-free-gamspy-available-now/7183) would present an alternative to using linopy/HiGHS. We would still have a python interface to the GAMS code, while possibly avoiding the actual rewriting of our GAMS code and likely avoiding the workflow problem presented by @SiddharthJoshi-Git below. The downside would be that our results would still not be completely publicly reproducible and we would still support the closed-source GAMS.
- @SiddharthJoshi-Git shared part of his usual workflow: for quick assessment of results or copying of data, he often uses the `.gdx` files, which seem to be much more performant than dumping the results to Excel or `.csv`. Our migration should take this into account and provide ways to quickly and conveniently access the same information stored in the `.gdx` as well as the `.log` and `.lst` files.
- @yiyi1991 expressed her support for this migration and stated that the [OseMOSYS team has already done something similar](https://github.com/OSeMOSYS/OSeMOSYS_linopy). This might be a place to look for examples as well as an opportunity for collaboration.
- @khaeru noted that we would need to ensure before migration that all GAMS features we use are also available with our choice of tools and that performance is at least at comparable levels.
All of this means that we're not likely to start this migration anytime soon, much less finish it, but I wanted to record this discussion. For future reference and for you, dear enthusiastic community member: please reach out to us to get information on the current status of this project! Any help will be greatly appreciated :heart:
Contributor guide
Research direction
Start by reviewing the ixmp4 transport tutorial proof of concept in PR 101 and the OSeMOSYS_linopy project mentioned in the issue. Inventory the GAMS features and the .gdx, .log, and .lst access patterns that must be retained, then compare the proposed replacement options. Done would require an agreed migration plan with comparable functionality and performance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100