[lldb] `LLDB_ENABLE_LIBEDIT=ON` incompatible with `LLDB_ENABLE_PYTHON_LIMITED_API=ON`
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
I'm [trying](https://github.com/conda-forge/lldb-feedstock/pull/110) to switch the LLDB builds in conda-forge to python's limited API, to avoid having to do multiple builds for each python version; it's very nice that lldb supports `LLDB_ENABLE_PYTHON_LIMITED_API=ON` already. FYI, I'm writing this issue from the POV of v22.1.8, but the same issues (and more) exist as of v23.1.0, so I want to first fix v22, where I already have everything in place, except the libedit support.
However, using the limited API support is incompatible with `LLDB_ENABLE_LIBEDIT=ON` (which the docs recommend to keep enabled), for a few reasons:
* the respective header does a bare include of `Python.h`, unguarded by the correct macros (e.g. setting `Py_LIMITED_API`)
https://github.com/llvm/llvm-project/blob/ca7933e47d3a3451d81e72ac174dcb5aa28b59d1/lldb/source/Plugins/ScriptInterpreter/Python/PythonReadline.h#L12-L18
* the [implementation](https://github.com/llvm/llvm-project/blob/llvmorg-22.1.8/lldb/source/Plugins/ScriptInterpreter/Python/PythonReadline.cpp) uses `PyMem_RawMalloc`, which was only [added](https://github.com/python/cpython/blob/v3.14.7/Doc/data/stable_abi.dat#L440) to the stable ABI in Python v3.13, whereas lldb currently defaults to the 3.8 API
https://github.com/llvm/llvm-project/blob/11a9127fbe072d84571712bb5e2aa9ebd964a802/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h#L35
* the implementation also uses `PyOS_ReadlineFunctionPointer`, which has been removed from the stable ABI as of https://github.com/python/cpython/commit/91b69b77cf5f78de6d35dea23098df34b6fd9e53 (v3.10), see https://github.com/python/cpython/issues/88034
The first two issues are pretty easy to work around, by a downstream-specific patch of the sort
```diff
--- a/lldb/source/Plugins/ScriptInterpreter/Python/PythonReadline.h
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/PythonReadline.h
@@ -9,14 +9,9 @@
#ifndef LLDB_PLUGINS_SCRIPTINTERPRETER_PYTHON_PYTHONREADLINE_H
#define LLDB_PLUGINS_SCRIPTINTERPRETER_PYTHON_PYTHONREADLINE_H
-#include "lldb/Host/Config.h"
+#include "lldb-python.h"
#if LLDB_ENABLE_LIBEDIT && defined(__linux__)
-// NOTE: Since Python may define some pre-processor definitions which affect the
-// standard headers on some systems, you must include Python.h before any
-// standard headers are included.
-#include "Python.h"
-
// no need to hack into Python's readline module if libedit isn't used.
//
#define LLDB_USE_LIBEDIT_READLINE_COMPAT_MODULE 1
diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h b/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h
index 5f61f4df8f99..2ebce36fbb4a 100644
--- a/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/lldb-python.h
@@ -45,7 +45,7 @@ static llvm::Expected *g_fcxx_modules_workaround [[maybe_unused]];
#include
#endif
-#define LLDB_MINIMUM_PYTHON_VERSION 0x03080000
+#define LLDB_MINIMUM_PYTHON_VERSION 0x030D0000
#if LLDB_ENABLE_PYTHON_LIMITED_API
// If defined, LLDB will be ABI-compatible with all Python 3 releases from the
```
However, the last issue is a hard blocker for me right now, and given that this seems to be a fundamental incompatibility between libedit support and using python's limited API, I wanted to ask what the view/intention of the lldb maintainers is on this. My tendency (for downstream) under the current circumstances would be to abandon the bits that require both python & libedit, because those bits are not worth quintupling our build matrix.
Regardless of the future direction, the two incompatible options should cause an error during configuration time, rather than "successfully" building to completion and then failing at runtime (when built against shared `libpython3.so`) with
```
+ lldb --version
lldb: symbol lookup error: $PREFIX/bin/../lib/liblldb.so.23.1: undefined symbol: _Py_Dealloc
```
Contributor guide
Research direction
Read lldb/source/Plugins/ScriptInterpreter/Python/PythonReadline.h, PythonReadline.cpp, and lldb-python.h, then trace how LLDB_ENABLE_LIBEDIT and LLDB_ENABLE_PYTHON_LIMITED_API are handled during configuration. The finished change should reject the incompatible option combination before building, rather than allowing a runtime undefined-symbol failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100