enthought / enthought/comtypes
Adapting to Python 3.15+ `IntFlag` changes for negative members.
- Dominant language
- Python
- Stars
- 345
- Forks
- 105
- PR merge metrics
- No merged PRs in 30d
Description
**To the community,**
In this project, `IntFlag` subclasses have been generated within friendly modules based on enumeration definitions retrieved from COM type libraries (see https://github.com/enthought/comtypes/issues/345).
Among these, some enums — such as `MsiInstallState` defined in the `Microsoft Windows Installer Object Library` — contain a mix of positive and negative values.
While these definitions worked as expected up to Python 3.14, the internal implementation of `IntFlag` is changing in Python 3.15.0-alpha, meaning they will no longer be evaluated as they were previously.
## Background
The root of this issue lies in the inconsistency of handling negative values within a flag (`IntFlag`).
As discussed in https://github.com/python/cpython/issues/107538 and https://github.com/python/cpython/pull/132273, `Flag` is intended to represent a "set of bits". Including negative values as members creates a logical contradiction.
I was the one who originally proposed and implemented the enum definitions in this project, and I should have recognized the irrationality of treating negative values as flags at that time.
I apologize for this oversight.
## Problem reproduction (minimal reproducer)
The following is a minimal reproducer using a snippet of the `MsiInstallState` definition generated by `import comtypes.client; comtypes.client.GetModule('msi.dll')`, along with a basic value check and test.
In Python 3.15.0-alpha and later, the evaluation of `IntFlag` is being changed, and negative members will no longer be evaluated as their literal values (e.g., `-1`).
```python
from enum import IntFlag
class MsiInstallState(IntFlag):
msiInstallStateNotUsed = -7
msiInstallStateBadConfig = -6
msiInstallStateIncomplete = -5
msiInstallStateSourceAbsent = -4
msiInstallStateInvalidArg = -2
msiInstallStateUnknown = -1
msiInstallStateBroken = 0
msiInstallStateAdvertised = 1
msiInstallStateRemoved = 1
msiInstallStateAbsent = 2
msiInstallStateLocal = 3
msiInstallStateSource = 4
msiInstallStateDefault = 5
if __name__ == "__main__":
print((repr(MsiInstallState.msiInstallStateUnknown), MsiInstallState.msiInstallStateUnknown.value))
assert MsiInstallState.msiInstallStateUnknown == -1
```
**Execution result in Python 3.14:**
```text
('', -1)
```
**Execution result in Python 3.15:**
```text
('', 7)
Traceback (most recent call last):
File "...\enum_testing.py", line 22, in
assert MsiInstallState.msiInstallStateUnknown == -1
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
```
### Why does the value become `7`?
In Python 3.15+, `IntFlag` treats only the bit range covered by the defined positive members as the **"valid bit domain"**.
Negative values are then reinterpreted as **"all bits ON (masked)"** within that specific range.
1. **Identifying the Valid Bit Range:**
The maximum positive value in `MsiInstallState` is `5` (`101` in binary).
Consequently, `IntFlag` determines the valid bit width for this enum to be the **lower 3 bits** ($2^0, 2^1, 2^2$).
2. **Masking the Negative Value:**
A negative value like `-1` has all bits set to `1` in its two's complement representation.
However, Python 3.15 `IntFlag` normalizes this to a state where **"all bits within the valid 3-bit range are `1`."**
3. **Calculation Result:**
With the lower 3 bits all set to `1`, the value becomes ($2^2 + 2^1 + 2^0 = 7$), losing consistency with the literal `-1`.
## Future direction
To ensure this project continues to function as intended in Python 3.15 and beyond, some form of adaptation is necessary.
At this stage, I want community feedback.
### Directions under consideration:
* **Direction 1: modification of code generation logic in `comtypes`**
* Adjust the generation method within the library to maintain value consistency even in Python 3.15+.
* **Direction 2: proposing changes to CPython regarding the handling of negative values**
* Demonstrate to the CPython community and the core developers that there is a demand for treating negative values as flags and advocate for improvements or maintenance of the Python<=3.14 behavior.
## Request to the community and discussion rules
Your feedback is vital.
This issue is something that all Python developers dealing with enums containing negative values may face — regardless of whether they use `comtypes`, or whether they define enums via dynamic generation or static hard-coding.
To ensure the discussion proceeds smoothly, please adhere to the following:
1. **If you have a specific proposal, please comment on this issue first.**
2. **Please do not create a pull request before discussing it here.** If discussions happen separately on PRs, it becomes difficult to track the decision-making process and hurts traceability.
If Python 3.15 reaches the release candidate (RC) stage and negative members are still not evaluated as literals, I intend to consider merging the only available proposal or the one deemed most rational at that time.
I look forward to a sincere discussion to find the best possible solution.
Thank you for your input.
Contributor guide
Assessment
This issue has not been assessed yet.