intel / intel/libva

[VAConfigAttribValEncHEVCBlockSizes] min/max_max_transform_hierarchy_depth_inter/intra don't have enough field bits

Open
#641 2 comments 0 reactions 1 assignee Claimed by @XinfengZhang View on GitHub
Dominant language
C
Stars
784
Forks
314
PR merge metrics
No merged PRs in 30d

Description

In the struct `VAConfigAttribValEncHEVCBlockSizes` defined in [here](https://github.com/intel/libva/blob/master/va/va_enc_hevc.h#L193), the min/max fields for `max_transform_hierarchy_depth_inter` and `max_transform_hierarchy_depth_intra` are 2 bits wide, but from the HEVC codec standard, both those values:

`...shall be in the range of 0 to CtbLog2SizeY − MinTbLog2SizeY, inclusive....`

When the hardware needs to report a configuration like below, the fields overflow to zero.

```
MinCbLog2SizeY = log2(8) = 3
CtbLog2SizeY = log2(64) = 6
MinTbLog2SizeY = log2(4) = 2
MaxTbLog2SizeY = log2(32) = 5

// The variable MinTbLog2SizeY is set equal to log2_min_luma_transform_block_size_minus2 + 2. The CVS shall not
contain data that result in MinTbLog2SizeY greater than or equal to MinCbLog2SizeY.
assert(MinTbLog2SizeY (2) < MinCbLog2SizeY (3));

max_transform_hierarchy_depth_intra = (CtbLog2SizeY − MinTbLog2SizeY) = 4
max_transform_hierarchy_depth_inter = (CtbLog2SizeY − MinTbLog2SizeY) = 4
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.