michaelfeil / michaelfeil/infinity
AMD MI50 does not work since version 0.069
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### System Info
return run_warmup(self, inp)
File "/app/infinity_emb/transformer/abstract.py", line 228, in run_warmup
embed = model.encode_core(feat)
File "/app/infinity_emb/transformer/crossencoder/torch.py", line 102, in encode_core
out_features = self.model(**features, return_dict=True)["logits"]
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 465, in _fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1269, in __call__
return self._torchdynamo_orig_callable(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1064, in __call__
result = self._inner_convert(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 526, in __call__
return _compile(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 924, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 666, in compile_inner
return _compile_inner(code, one_graph, hooks, transform)
File "/app/.venv/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrapper_function
return function(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 699, in _compile_inner
out_code = transform_code_object(code, transform)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object
transformations(instructions, code_options)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 219, in _fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 634, in transform
tracer.run()
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2796, in run
super().run()
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 983, in run
while self.step():
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 895, in step
self.dispatch_table[inst.opcode](self, inst)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2987, in RETURN_VALUE
self._return(inst)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2972, in _return
self.output.compile_subgraph(
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1142, in compile_subgraph
self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1369, in compile_and_call_fx_graph
compiled_fn = self.call_user_compiler(gm)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1416, in call_user_compiler
return self._call_user_compiler(gm)
File "/app/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 1465, in _call_user_compiler
raise BackendCompilerFailed(self.compiler_fn, e) from e
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
SubprocException: An exception occurred in a subprocess:
Traceback (most recent call last):
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/compile_worker/subproc_pool.py", line 270, in do_job
result = job()
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/runtime/compile_tasks.py", line 68, in _worker_compile_triton
load_kernel().precompile(warm_cache_only=True)
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/runtime/triton_heuristics.py", line 244, in precompile
compiled_binary, launcher = self._precompile_config(
File "/app/.venv/lib/python3.10/site-packages/torch/_inductor/runtime/triton_heuristics.py", line 428, in _precompile_config
triton.compile(*compile_args, **compile_kwargs),
File "/app/.venv/lib/python3.10/site-packages/triton/compiler/compiler.py", line 282, in compile
next_module = compile_ir(module, metadata)
File "/app/.venv/lib/python3.10/site-packages/triton/backends/amd/compiler.py", line 255, in
stages["llir"] = lambda src, metadata: self.make_llir(src, metadata, options)
File "/app/.venv/lib/python3.10/site-packages/triton/backends/amd/compiler.py", line 186, in make_llir
pm.run(mod)
RuntimeError: PassManager::run failed
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
You can suppress this exception and fall back to eager by setting:
import torch._dynamo
torch._dynamo.config.suppress_errors = True
ERROR: Application startup failed. Exiting.
loc("/tmp/torchinductor_root/fs/cfs7uexc7lhcuuycwi7c4tsdl5lqblkxiybraqvyr4qvpnczxfq3.py":18:0): error: unsupported target: 'gfx906'
loc("/tmp/torchinductor_root/rb/crbz636dzyyhj5fhqjyt57crcfntltxcnf6sb325blep45efcgx7.py":18:0): error: unsupported target: 'gfx906'
loc("/tmp/torchinductor_root/fu/cfu3mm7vubttwkr7jqp6akoon6hxbdhmvagd5vmunf4pbtx6evt3.py":18:0): error: unsupported target: 'gfx906'
loc("/tmp/torchinductor_root/ig/cigio5uq7soptxet4jnfr5xuv76ne4dbhb3n3x3koyb7eqofa2ez.py":18:0): error: unsupported target: 'gfx906'
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO 2025-05-29 00:49:35,230 infinity_emb INFO: infinity_server.py:89
Creating 1engines:
engines=['BAAI/bge-reranker-v2-m3']
INFO 2025-05-29 00:49:35,239 infinity_emb INFO: Anonymized telemetry.py:30
telemetry can be disabled via environment variable
`DO_NOT_TRACK=1`.
INFO 2025-05-29 00:49:35,256 infinity_emb INFO: select_model.py:64
model=`BAAI/bge-reranker-v2-m3` selected, using
engine=`torch` and device=`None`
INFO 2025-05-29 00:49:40,535 infinity_emb INFO: using torch.py:84
torch.compile(dynamic=True)
### Information
- [x] Docker + cli
- [ ] pip + cli
- [ ] pip + usage of Python interface
### Tasks
- [x] An officially supported CLI command
- [ ] My own modifications
### Reproduction
When using version 0.069 and above:
services:
infinity_server:
image: michaelf34/infinity:0.0.69-amd
container_name: infinity_server
cap_add:
- SYS_PTRACE
devices:
- "/dev/kfd:/dev/kfd"
- "/dev/dri:/dev/dri"
ports:
- "7997:7997"
volumes:
- ./data:/app/.cache
restart: unless-stopped
command: >
v2
--model-id BAAI/bge-reranker-v2-m3
--engine torch
--compile
--no-bettertransformer
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the provided Docker Compose configuration and compare versions 0.068 and 0.069. Start with transformer/abstract.py, transformer/crossencoder/torch.py, and the torch.compile path shown in the traceback; inspect the AMD Triton error for target gfx906. Done means the AMD MI50 container starts successfully with the documented command.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100