Macaulay2 / Macaulay2/M2

SMat subtraction skips the last coefficient and crashes on empty columns

Open Beginner friendly
#4,707 0 comments 0 reactions 0 assignees View on GitHub
Engine
Dominant language
Macaulay2
Stars
435
Forks
297
Avg merge
4d 20h
Merged PRs (30d)
11

Description

`SMat::subtractInPlace` uses `vec_negate` in `Macaulay2/e/basic-mutable-matrices/smat.hpp`. Its loop starts with `p = v` and tests `p->next != NULL`, which skips the last stored coefficient and dereferences null when the input column is empty.

Reproduced with `SMat` over ZZ/101:

1. Subtracting the column `[2, 3]` from `[7, 11]` produces `[9, 8]`, rather than `[5, 8]` (columns listed top to bottom). The lowest stored row is added instead of subtracted.
2. Subtracting a zero column from `[7, 11]` terminates with SIGSEGV (exit 139), rather than leaving the column unchanged.

Minimal C++ reproduction (inside the engine unit-test harness):

```cpp
M2::ARingZZp ring(101);
SMat left(ring, 2, 1), right(ring, 2, 1);
M2::ARingZZp::Element value(ring);
ring.set(value, 7);
left.set_entry(0, 0, value);
ring.set(value, 11);
left.set_entry(1, 0, value);
// Leave right empty to reproduce the crash. Alternatively, set its
// entries to 2 and 3 to reproduce the incorrect result [9, 8].
left.subtractInPlace(right);
```

These were reproduced with a Debug CMake engine build on macOS arm64, with no production changes. `vec_negate` needs to visit every non-null node and accept an empty list. This was found while adding dedicated SMat engine unit tests.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in Macaulay2/e/basic-mutable-matrices/smat.hpp at SMat::subtractInPlace and its vec_negate loop. Reproduce both cases in the engine unit-test harness using the ZZ/101 example: subtraction must include the last coefficient, and an empty input column must not crash. Run the dedicated SMat engine tests and confirm both results.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.