deepjavalibrary / deepjavalibrary/djl

DJL对于FP16模型推理的支持情况

Open
#2,571 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
4.9k
Forks
759
Avg merge
16h 15m
Merged PRs (30d)
15

Description

## Description
目前djl的Image及内置的相关Translator默认都是FP32精度,查看相关源码及API发现并未找到相关的设置用来支持FP16模型的推理,FP16精度的模型在损失一定的精度下可以提高推理速度,希望后续版本能够提供相关API或者是设置项用来快速的实现FP16模型的推理。

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.