flutter / flutter/flutter

[Impeller] Decoded greyscale images cost 3.7x to 4x in memory what they cost under Skia

Open
#190,573 2 comments 0 reactions 0 assignees View on GitHub
a: images c: performance e: impeller engine found in release: 3.44 P2 team-engine triaged-engine
Dominant language
Dart
Stars
179k
Forks
31.1k
PR merge metrics
PR metrics pending

Description

Under Impeller, a decoded 1-component (greyscale) JPEG occupies the same GPU
memory as the same image decoded from a 3-component JPEG. Under Skia it
occupies about a quarter.

Galaxy Tab S8+, Flutter 3.44.8 stable, profile build. 64 copies of one 768x512
image, held and drawn:

| Renderer | 64 x 1-component JPEG | 64 x 3-component JPEG |
|---|---|---|
| Skia | **36.2 MB** (1.51 B/px) | 138.9 MB (5.79 B/px) |
| Impeller | **135.3 MB** (5.64 B/px) | not separable, see below |

Graphics memory from `dumpsys meminfo`, delta against the baseline in the same
run. 64 x 768 x 512 = 25.2 Mpx. Two runs per renderer, agreeing to within 12 kB.

Impeller did not release the greyscale allocations on `dispose()`, and the RGB
set then fit inside the pool already claimed, so its own footprint is not
readable in the same run. The greyscale figure already equals Skia's RGB figure.

## Repro

Assets. `kodim23` from the [Kodak Lossless True Color Image
Suite](https://r0k.us/graphics/kodak/), 768x512, encoded twice from one source
so component count is the only variable:

```python
from PIL import Image
im = Image.open('kodim23.png')
im.convert('L').save('assets/kodim23_grey.jpg', quality=90) # 1 component
im.convert('RGB').save('assets/kodim23_rgb.jpg', quality=90) # 3 components
```

`lib/main.dart`:

```dart
import 'dart:ui' as ui;
import 'package:flutter/material.dart';
import 'package:flutter/services.dart' show rootBundle;

const int kN = 64;
final List _held = [];
final ValueNotifier _repaint = ValueNotifier(0);

Future holdN(String asset) async {
final bytes = (await rootBundle.load(asset)).buffer.asUint8List();
for (int i = 0; i < kN; i++) {
// No ImageProvider: nothing comes from the image cache, every image is a
// distinct allocation.
final buffer = await ui.ImmutableBuffer.fromUint8List(bytes);
final d = await ui.ImageDescriptor.encoded(buffer);
final codec = await d.instantiateCodec();
_held.add((await codec.getNextFrame()).image);
codec.dispose(); d.dispose(); buffer.dispose();
}
_repaint.value++;
await WidgetsBinding.instance.endOfFrame;
await WidgetsBinding.instance.endOfFrame; // uploaded, not deferred
print('MARK drawn $asset n=${_held.length}');
}

void freeAll() {
for (final i in _held) { i.dispose(); }
_held.clear();
_repaint.value++;
print('MARK freed');
}

class _Grid extends CustomPainter {
_Grid() : super(repaint: _repaint);
@override
void paint(Canvas canvas, Size size) {
final cw = size.width / 8, ch = size.height / 8;
for (int i = 0; i < _held.length; i++) {
final img = _held[i];
canvas.drawImageRect(
img,
Rect.fromLTWH(0, 0, img.width.toDouble(), img.height.toDouble()),
Rect.fromLTWH((i % 8) * cw, (i ~/ 8) * ch, cw, ch),
Paint());
}
}
@override
bool shouldRepaint(covariant _Grid old) => true;
}

Future probe() async {
print('MARK baseline');
await Future.delayed(const Duration(seconds: 12));
for (final a in ['assets/kodim23_grey.jpg', 'assets/kodim23_rgb.jpg']) {
await holdN(a);
await Future.delayed(const Duration(seconds: 14));
freeAll();
await Future.delayed(const Duration(seconds: 12));
}
}

void main() {
WidgetsFlutterBinding.ensureInitialized();
probe();
runApp(MaterialApp(home: Scaffold(
backgroundColor: Colors.white,
body: CustomPaint(painter: _Grid(), size: Size.infinite))));
}
```

```
flutter build apk --profile
adb shell am start -n /.MainActivity --ez enable-impeller true
adb shell am start -n /.MainActivity --ez enable-impeller false
adb shell dumpsys meminfo | grep Graphics # sample at each MARK
```

## The Dart API reports no difference

Same asset, same device, both renderers:

```
assets/kodim23_grey.jpg 768x512 descriptor.bytesPerPixel=1
toByteData(rawUnmodified) = 4.00 B/px
```

`bytesPerPixel` reports the source component count. `rawUnmodified` returns
4 B/px under Impeller and under Skia, while the GPU footprint differs by 3.7x.
Application code cannot detect the difference.

## Impeller's texture path does store 1 B/px

Same device, same measurement, `flutter_gpu`, 64 textures of 1024x1024:

| Allocation | Declared | Graphics delta |
|---|---|---|
| `r8UNormInt` | 67.1 MB | +71.8 MB |
| `r8g8b8a8UNormInt` | 268.4 MB | +199.7 MB, plus the 71.8 MB pool reused |

Single-channel storage holds through `gpu.Texture`. The codec path does not use
it.

## Impact

A document viewer caching rendered pages as JPEG and decoding them through
`instantiateCodec`. Scanned pages encode as 1-component JPEGs. Toggling that
encoder between 1-component and 3-component output, 529-page scanned document,
cold, same gestures, same scroll distance:

| Greyscale encoding | Impeller PSS | Skia PSS |
|---|---|---|
| on | 548 MB | 406 MB |
| off | 539 MB | 465 MB |

Skia rises 59 MB when greyscale output is off. Impeller does not move. The
`--no-enable-impeller` opt-out is deprecated; on this content it is worth about
70 MB.

## Environment

```
Flutter 3.44.8 • channel stable
Framework • revision 058e0af2c2 (2026-07-23)
Engine • hash 13ffd72b2f9a5ca4db2a74ea52d5353ec2e8f939 (revision 0cd610717b)
Device: Samsung Galaxy Tab S8+ (SM-X806B), Android 14, Vulkan
```

## Related

[#103171](https://github.com/flutter/flutter/issues/103171) requests `rgb565`
and `rgb888` in `ui.PixelFormat`. Separate: no new API is involved here, Skia
already stores these images compactly through the existing one.

Contributor guide

Open the contributing guide

Research direction

Start with the reproduction in lib/main.dart, especially ui.ImageDescriptor.encoded, instantiateCodec, and the dumpsys meminfo measurements; compare that codec path with the flutter_gpu r8UNormInt allocation. Done means 1-component decoded images no longer consume the same GPU memory as 3-component images and are released after dispose().

Written by the indexing model from the issue text.

Assessment

Tech stack
android, dart, flutter
Domain
computer-graphics, mobile-dev, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.