chai2010 / chai2010/webp

memory leak

Open
#26 5 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
Go
Stars
723
Forks
113
PR merge metrics
No merged PRs in 30d

Description

I create a simple http server do convert a jpg to webp

here is the sample code:

```
func ImgTowebp(imgData []byte) ([]byte, error) {
//jpeg.Decode()
img, err := jpeg.Decode(bytes.NewBuffer(imgData))
if err != nil {
//log.Println("decode error from jpg",err)
img, err = png.Decode(bytes.NewBuffer(imgData))
if err != nil {
log.Println("decode error from png or jpg", err)
return nil, err
}
}
//log.Println(fname)
webImgBytes, err := webp.EncodeRGBA(img, 80)
if err == nil {
webpLength := len(webImgBytes)
imgLength := len(imgData)
if webpLength > imgLength {
log.Println("webp bigger than img")
}
log.Println("img2webp :", webpLength, ":", imgLength)
return webImgBytes, nil
} else {
log.Println("webp encode jpg file", err)
return nil, err
}

}

```

main.go
```
webImgBytes, err := ImgTowebp(bodybuffer)
if err == nil {
w.WriteHeader(http.StatusOK)
w.Header().Set("Content-Type", "image/webp")
w.Write(webImgBytes)
return
}
```

运行一段时间后, 内存分析图:

![20191217141007](https://user-images.githubusercontent.com/492946/70969575-fdb7e900-20d6-11ea-9111-daf4dd2522de.jpg)

查看 相关源代码:

```
func EncodeRGBA(m image.Image, quality float32) (data []byte, err error) {
p := toRGBAImage(m)
data, err = webpEncodeRGBA(p.Pix, p.Rect.Dx(), p.Rect.Dy(), p.Stride, quality)
return
}

func toRGBAImage(m image.Image) *image.RGBA {
if m, ok := m.(*image.RGBA); ok {
return m
}
b := m.Bounds()
rgba := image.NewRGBA(b)
dstColorRGBA64 := &color.RGBA64{}
dstColor := color.Color(dstColorRGBA64)
for y := b.Min.Y; y < b.Max.Y; y++ {
for x := b.Min.X; x < b.Max.X; x++ {
pr, pg, pb, pa := m.At(x, y).RGBA()
dstColorRGBA64.R = uint16(pr)
dstColorRGBA64.G = uint16(pg)
dstColorRGBA64.B = uint16(pb)
dstColorRGBA64.A = uint16(pa)
rgba.Set(x, y, dstColor)
}
}
return rgba
}

func webpEncodeRGB(pix []byte, width, height, stride int, quality float32) (output []byte, err error) {
if len(pix) == 0 || width <= 0 || height <= 0 || stride <= 0 || quality < 0.0 {
err = errors.New("webpEncodeRGB: bad arguments")
return
}
if stride < width*3 && len(pix) < height*stride {
err = errors.New("webpEncodeRGB: bad arguments")
return
}

//主要怀疑 (*C.uint8_t)(unsafe.Pointer(&pix[0])) 这个是golang分配的图片pix数组, 传递到CGO后,没有进行释放,就一直持有。
///
var cptr_size C.size_t
var cptr = C.webpEncodeRGB(
(*C.uint8_t)(unsafe.Pointer(&pix[0])), C.int(width), C.int(height),
C.int(stride), C.float(quality),
&cptr_size,
)
if cptr == nil || cptr_size == 0 {
err = errors.New("webpEncodeRGB: failed")
return
}
defer C.free(unsafe.Pointer(cptr))

output = make([]byte, int(cptr_size))
copy(output, ((*[1 << 30]byte)(unsafe.Pointer(cptr)))[0:len(output):len(output)])
return
}

```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the ImgTowebp example in the issue and trace the path through main.go, EncodeRGBA, toRGBAImage, and webpEncodeRGB. Reproduce the HTTP server workload and inspect the memory profile while checking ownership across the C call and the returned byte slice. Done means the reported memory growth is explained and the conversion no longer retains allocations unexpectedly.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, go
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.