intel / intel/intel-lpmd

Native IRQ affinity is not restored after the first cycle in multi-state configurations

Open
#126 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
223
Forks
32
Avg merge
1m
Merged PRs (30d)
1

Description

Without `irqbalance` , intel-lpmd uses its native IRQ migration path. With a configuration containing the following states:

| State | Active CPUs | `IRQMigrate` |
| --- | --- | --- |
| `ECORE_ONE_MODULE` | `16-19` | `1` |
| `ECORE_TWO_MODULES` | `12-19` | `1` |
| `FULL_POWER` | `0-19` | `-2` (`SETTING_RESTORE`) |

In a multi-state cycle, IRQ affinity is restored correctly only the first time.

During the first transition to a restricted state, the native implementation saves each IRQ's original `smp_affinity` while [`irq_updated` is false](https://github.com/intel/intel-lpmd/blob/40d18a6cc22c37addc3e636bc9c5cf1ab5d5fbda/src/lpmd_irq.c#L91-L131). After updating the IRQs, it [sets `irq_updated` to `1`](https://github.com/intel/intel-lpmd/blob/40d18a6cc22c37addc3e636bc9c5cf1ab5d5fbda/src/lpmd_irq.c#L134-L194).

When entering `FULL_POWER`, `IRQMigrate=-2` selects the native restore path. [`native_restore_irqs()` restores the cached affinities and clears the cache with `memset()`](https://github.com/intel/intel-lpmd/blob/40d18a6cc22c37addc3e636bc9c5cf1ab5d5fbda/src/lpmd_irq.c#L71-L86), but it does not reset `irq_updated` to `0`.

Consequently, during the next low-power cycle, `irq_updated` is still `1`, so the original IRQ affinities are not cached again. The IRQs are restricted to the selected cluster, but the following `FULL_POWER` transition has an empty cache and therefore restores nothing. The state machine still applies the `FULL_POWER` CPU configuration after processing IRQs, because [`process_cgroup()` is called after `process_irq()`](https://github.com/intel/intel-lpmd/blob/40d18a6cc22c37addc3e636bc9c5cf1ab5d5fbda/src/lpmd_state_machine.c#L320-L337).

For example, after the second `ECORE_TWO_MODULES -> FULL_POWER` transition, the system can end up in the following inconsistent state:

```text
CPU affinity: 0-19
IRQ affinity: 12-19
```

If the transition passes through `ECORE_ONE_MODULE`, IRQ affinity may remain restricted to `16-19` instead.

Expected behavior: every `FULL_POWER` transition with `IRQMigrate=-2` restores the IRQ affinities that existed before the corresponding low-power cycle.

Actual behavior: only the first restore succeeds; subsequent restores leave IRQ affinity restricted to the last low-power cluster.

***

```xml


-1
1
-1
0
1
0
0
10
95
1


6
186
*

1
ECORE_ONE_MODULE
2
50

16-19

-1
-1
-1
1

500
-1
2000


2
ECORE_TWO_MODULES
10
95

12-19

-1
-1
-1
1

500
-1
2000


3
FULL_POWER

0-19

-1
-1
-1

-2

500
-1
2000

```

Contributor guide

Open the contributing guide

Research direction

Start in src/lpmd_irq.c, focusing on native_restore_irqs() and the irq_updated flag used around IRQ affinity caching and restoration. Review the transition ordering in src/lpmd_state_machine.c, then reproduce repeated cycles with the supplied multi-state configuration. Done means each FULL_POWER transition restores the affinities saved during its preceding low-power cycle.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.