apache / apache/tvm

[Bug] Inconcistency after meta schedule for the conv2d_hwcn operator

Open
#17,997 3 comments 0 reactions 0 assignees View on GitHub
needs-triage type: bug
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 1h
Merged PRs (30d)
112

Description

### Actual behavior

```
An Inconsistency bug detected.
Mismatched elements: 4093 / 294912 (1.39%)
Max absolute difference among violations: 0.00019836
Max relative difference among violations: 0.25589287
ACTUAL: array([[[[ 0.653972, 4.73613 , 47.223606, ..., 1.366684,
-10.288207, 2.612887],
[ 8.710161, -28.032656, 5.167299, ..., -5.342846,...
DESIRED: array([[[[ 0.653983, 4.736097, 47.22358 , ..., 1.366686,
-10.288215, 2.612871],
[ 8.710163, -28.032671, 5.167283, ..., -5.342826,...

```

### Environment

tvm-v0.21.dev0

### Steps to reproduce

```
import tvm
from tvm import te, topi, tir
from tvm import meta_schedule as ms
import numpy as np

def compile_mod(mod, np_input_list, output_shape, output_type, opt_level=3):
with tvm.transform.PassContext(opt_level):
ref_mod = tvm.build(mod, target='llvm')
mod_output = tvm.nd.empty(output_shape, dtype=output_type, device=tvm.cpu(0))

tvm_inputs = [tvm.nd.array(x) for x in np_input_list]
ref_mod(*tvm_inputs, mod_output)
return mod_output

Input = te.placeholder([7, 7, 512, 32], dtype='float32', name='Input')
Filter = te.placeholder([3, 3, 512, 1024], dtype='float32', name='Filter')
op_config = {'Input': Input, 'Filter': Filter, 'stride': [1, 1], 'padding': [1, 1], 'dilation': [3, 3], }
op_output = topi.nn.conv2d_hwcn(**op_config)
np_inputs = [np.random.uniform(-1, 1, size=[7, 7, 512, 32]).astype('float32'),np.random.uniform(-1, 1, size=[3, 3, 512, 1024]).astype('float32')]
sch = tir.Schedule(te.create_prim_func([Input, Filter, op_output]).with_attr('target', tvm.target.Target('llvm')))
ref_output = compile_mod(sch.mod, np_inputs, op_output.shape, op_output.dtype, opt_level=0)

database = ms.tir_integration.tune_tir(mod=sch.mod, target='llvm --num-cores=16', work_dir='./tune_tmp', max_trials_global=1, num_trials_per_iter=1)
sch = ms.tir_integration.compile_tir(database, sch.mod, 'llvm --num-cores=16')

opt_mod_output = compile_mod(sch.mod, np_inputs, op_output.shape, op_output.dtype, opt_level=4)

np.testing.assert_allclose(
ref_output.numpy(), opt_mod_output.numpy(), rtol=1e-5, atol=1e-5, err_msg=f"An Inconsistency detected."
)

```

### Triage

* needs-triage

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the provided Python reproduction with TVM v0.21.dev0 and compare the unoptimized LLVM output with the meta-schedule-tuned output. Trace the generated schedule for topi.nn.conv2d_hwcn and investigate the numerical mismatch; done means the tuned result satisfies the stated allclose tolerance without regressing the reference result.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
compilers, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.