michaelfeil / michaelfeil/infinity

AMD MI50 does not work since version 0.069

Open
#593 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
206
PR merge metrics
No merged PRs in 30d

Description

### System Info

return run_warmup(self, inp)
File "/app/infinity_emb/transformer/abstract.py", line 228, in run_warmup
embed = model.encode_core(feat)
File "/app/infinity_emb/transformer/crossencoder/torch.py", line 102, in encode_core
out_features = self.model(**features, return_dict=True)["logits"]
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 465, in _fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1269, in __call__
return self._torchdynamo_orig_callable(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1064, in __call__
result = self._inner_convert(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 526, in __call__
return _compile(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 924, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 666, in compile_inner
return _compile_inner(code, one_graph, hooks, transform)
File "/app/.venv/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrapper_function
return function(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 699, in _compile_inner
out_code = transform_code_object(code, transform)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object
transformations(instructions, code_options)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 219, in _fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 634, in transform
tracer.run()
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2796, in run
super().run()
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 983, in run
while self.step():
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 895, in step
self.dispatch_table[inst.opcode](self, inst)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2987, in RETURN_VALUE
self._return(inst)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2972, in _return
self.output.compile_subgraph(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1142, in compile_subgraph
self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1369, in compile_and_call_fx_graph
compiled_fn = self.call_user_compiler(gm)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1416, in call_user_compiler
return self._call_user_compiler(gm)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1465, in _call_user_compiler
raise BackendCompilerFailed(self.compiler_fn, e) from e
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
SubprocException: An exception occurred in a subprocess:
Traceback (most recent call last):
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 270, in do_job
result = job()
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/runtime/compile_tasks.py", line 68, in _worker_compile_triton
load_kernel().precompile(warm_cache_only=True)
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/runtime/triton_heuristics.py", line 244, in precompile
compiled_binary, launcher = self._precompile_config(
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/runtime/triton_heuristics.py", line 428, in _precompile_config
triton.compile(*compile_args, **compile_kwargs),
File "/app/.venv/lib/python3.10/site-packages/triton/compiler/compiler.py", line 282, in compile
next_module = compile_ir(module, metadata)
File "/app/.venv/lib/python3.10/site-packages/triton/backends/amd/compiler.py", line 255, in
stages["llir"] = lambda src, metadata: self.make_llir(src, metadata, options)
File "/app/.venv/lib/python3.10/site-packages/triton/backends/amd/compiler.py", line 186, in make_llir
pm.run(mod)
RuntimeError: PassManager::run failed
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
ERROR: Application startup failed. Exiting.
loc("/tmp/torchinductor_root/fs/cfs7uexc7lhcuuycwi7c4tsdl5lqblkxiybraqvyr4qvpnczxfq3.py":18:0): error: unsupported target: 'gfx906'
loc("/tmp/torchinductor_root/rb/crbz636dzyyhj5fhqjyt57crcfntltxcnf6sb325blep45efcgx7.py":18:0): error: unsupported target: 'gfx906'
loc("/tmp/torchinductor_root/fu/cfu3mm7vubttwkr7jqp6akoon6hxbdhmvagd5vmunf4pbtx6evt3.py":18:0): error: unsupported target: 'gfx906'
loc("/tmp/torchinductor_root/ig/cigio5uq7soptxet4jnfr5xuv76ne4dbhb3n3x3koyb7eqofa2ez.py":18:0): error: unsupported target: 'gfx906'
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO 2025-05-29 00:49:35,230 infinity_emb INFO: infinity_server.py:89
Creating 1engines:
engines=['BAAI/bge-reranker-v2-m3']
INFO 2025-05-29 00:49:35,239 infinity_emb INFO: Anonymized telemetry.py:30
telemetry can be disabled via environment variable
`DO_NOT_TRACK=1`.
INFO 2025-05-29 00:49:35,256 infinity_emb INFO: select_model.py:64
model=`BAAI/bge-reranker-v2-m3` selected, using
engine=`torch` and device=`None`
INFO 2025-05-29 00:49:40,535 infinity_emb INFO: using torch.py:84
torch.compile(dynamic=True)

### Information

- [x] Docker + cli
- [ ] pip + cli
- [ ] pip + usage of Python interface

### Tasks

- [x] An officially supported CLI command
- [ ] My own modifications

### Reproduction

When using version 0.069 and above:
services:
infinity_server:
image: michaelf34/infinity:0.0.69-amd
container_name: infinity_server
cap_add:
- SYS_PTRACE
devices:
- "/dev/kfd:/dev/kfd"
- "/dev/dri:/dev/dri"
ports:
- "7997:7997"
volumes:
- ./data:/app/.cache
restart: unless-stopped
command: >
v2
--model-id BAAI/bge-reranker-v2-m3
--engine torch
--compile
--no-bettertransformer

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the provided Docker Compose configuration and compare versions 0.068 and 0.069. Start with transformer/abstract.py, transformer/crossencoder/torch.py, and the torch.compile path shown in the traceback; inspect the AMD Triton error for target gfx906. Done means the AMD MI50 container starts successfully with the documented command.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python, pytorch
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.