dart-lang / dart-lang/native

CGImageRef gets released before the listener is called

Open
#1,467 7 comments 0 reactions 1 assignee Claimed by @liamappelbe View on GitHub
lang-objective_c package:ffigen
Dominant language
Dart
Stars
275
Forks
144
Avg merge
2d 10h
Merged PRs (30d)
47

Description

Hi!

I'm trying to use the `AVAssetImageGenerator.generateCGImagesAsynchronouslyForTimes_completionHandler` via ffigen to extract video frames (e.g. the thumbnail). For some reason, when the completion handler is called, in some situations (not always!) it seems like the CGImageRef is already released when the listener is called. Here's a shortened piece of code that shows the usage from Dart:

```dart
Future> _extractFramesAt(
VideoSource source, {
required List timestamps,
ui.Size? preferredSize,
}) async {
return using>>((alloc) async {
// Prepare the asset image generator
final avAsset = videoSourceToAVAsset(source);
final generator = AVAssetImageGenerator.assetImageGeneratorWithAsset_(avAsset);
generator.appliesPreferredTrackTransform = true;

// Omitted...

// This closure is called every time an image is generated
final closure = ObjCBlock_ffiVoid_CMTime_CGImage_CMTime_AVAssetImageGeneratorResult_NSError.listener(
(requestedTime, cgImagePointer, actualTime, result, error) {
if (error != null) {
imageStream.addError(error);
return;
}

if (cgImagePointer == nullptr) {
imageStream.addError(Exception('No image generated'));
return;
}

// Paint the image onto a CGBitmap
final width = darwinLib.CGImageGetWidth(cgImagePointer);
final height = darwinLib.CGImageGetHeight(cgImagePointer);
final colorSpace = darwinLib.CGColorSpaceCreateDeviceRGB();

if (width == 0 || height == 0) { // <- This gets called often with both width and height equal to 0.
imageStream.addError(Exception('Invalid image size: $width x $height'));
return;
}

// Omitted...
},
);

// Run the generator. The closure will be called for each timestamp
generator.generateCGImagesAsynchronouslyForTimes_completionHandler_(
cmTimeArray,
closure,
);

// Omitted...
});
}
```

From what I can understand, somehow the CGImageRef is getting released too early, before the listener is called. I was able to consistently replicate this on my phone. Also, I was able to fix this in the generated code by adding an explicit `CGImageRetain` call to the listener block:

```objc
// Before
typedef void (^ListenerBlock51)(CMTime , struct CGImage * , CMTime , AVAssetImageGeneratorResult , NSError* );
ListenerBlock51 wrapListenerBlock_ObjCBlock_ffiVoid_CMTime_CGImage_CMTime_AVAssetImageGeneratorResult_NSError(ListenerBlock51 block) NS_RETURNS_RETAINED {
return ^void(CMTime arg0, struct CGImage * arg1, CMTime arg2, AVAssetImageGeneratorResult arg3, NSError* arg4) {
block(arg0, arg1, arg2, arg3, objc_retain(arg4));
};
}

// After
typedef void (^ListenerBlock51)(CMTime , struct CGImage * , CMTime , AVAssetImageGeneratorResult , NSError* );
ListenerBlock51 wrapListenerBlock_ObjCBlock_ffiVoid_CMTime_CGImage_CMTime_AVAssetImageGeneratorResult_NSError(ListenerBlock51 block) NS_RETURNS_RETAINED {
return ^void(CMTime arg0, struct CGImage * arg1, CMTime arg2, AVAssetImageGeneratorResult arg3, NSError* arg4) {
CGImageRetain(arg1); // <- Added this
block(arg0, arg1, arg2, arg3, objc_retain(arg4));
};
}

```

and then releasing the CGImage on the Dart side. This seems to fix the issue, but I'm not sure whether it should be that way.

If needed, I can provide an example project which should be able to reproduce this issue. I'm using ffigen/objc from https://github.com/dart-lang/native/pull/1463

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.